TW202234886A - Adaptive loop filter with fixed filters - Google Patents

Abstract

A video decoder can be configured to apply a first stage adaptive loop filter (ALF) to a reconstructed sample by determining a first class index for the reconstructed sample, selecting a filter from a first set of filters based on the first class index, and applying the filter from the first set of filters to the reconstructed sample to determine a first intermediate sample value; apply a second stage ALF to the reconstructed sample by determining a second class index for the reconstructed sample; select a second filter from a second set of filters based on the second class index, applying the second filter to the reconstructed sample to determine a first sample modification value, and determining a second sample modification value based on the first intermediate sample value; and determine a filtered reconstructed sample based on the reconstructed sample and the first and second sample modification values.

Description

Self-tuning loop filter with fixed filter

This patent application claims the following rights: U.S. Provisional Patent Application 63/148,538, filed on February 11, 2021; and U.S. Provisional Patent Application 63/130,275, filed December 23, 2020, The entire contents of each of the aforementioned US Provisional Patent Applications are incorporated herein by reference.

The content of this case is about video encoding and video decoding.

Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital broadcast systems, wireless broadcasting systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers devices, digital cameras, digital recording equipment, digital media players, video game devices, video game consoles, cellular or satellite radio telephones (so-called "smart phones"), video teleconferencing equipment, video streaming equipment, etc. Digital video equipment implementing video coding techniques (such as those described in MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4 (Part 10, Advanced Video Coding (AVC)) , the standards defined by ITU-T H.265/High Efficiency Video Coding (HEVC) and those techniques described in extensions to such standards). By implementing such video decoding techniques, video devices can transmit, receive, encode, decode and/or store digital video information more efficiently.

Video coding techniques include spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (eg, a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or decoding node. Video blocks in an intra-frame decoded (I) slice of a picture are encoded using spatial prediction relative to reference samples in neighboring blocks in the same picture. Video blocks in inter-frame decoded (P or B) slices of a picture may use spatial prediction relative to reference samples in neighboring blocks in the same picture or temporal prediction relative to reference samples in other reference pictures. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame.

In accordance with the techniques disclosed herein, a video decoder may be configured to perform two-stage self-adjusting loop filtering. For the first stage ALF, the video decoder may be configured to determine activity and orientation values for the reconstructed samples being filtered. The activity value may generally indicate a variation of sample values in the sample neighborhood around the filtered sample. The direction value may generally indicate the direction in which the sample value is changing, eg, whether the sample value is increasing horizontally, vertically, at 45 degrees, at 135 degrees, or not at all. Based on the activity value and the direction value, the video decoder can determine the class index of the filtered samples. A class index may be associated with a filter from the first set of filters, and each filter is defined by the shape and set of coefficient values. The video decoder can decide the first set of filters and which classifiers map to which filters based on the syntax included in the bitstream. The video decoder may apply a filter to the reconstructed samples to determine the first intermediate sample value.

For the second stage ALF, the video decoder determines a first sample modification value by applying a second filter to the reconstructed samples, and determines a second sample modification value based on the first intermediate sample value. The video decoder then determines the filtered reconstructed samples based on the reconstructed samples, the first sample modification value and the second sample modification value. By applying the first-stage ALF to the reconstructed samples of the reconstruction block and the second-stage ALF to the reconstructed samples, as described herein, the techniques of the present case can do better than single-stage ALF by taking into account the local features of the video data, to improve the overall quality of the decoded video data.

According to one example, a method of decoding video data includes applying a first stage self-adjusting loop filter (ALF) to reconstructed samples of a reconstruction block, wherein applying the first stage ALF includes determining a first stage for reconstructing samples class index; selecting a filter from the first set of filters based on the first class index; and applying the filter from the first set of filters to the reconstructed samples to determine the first intermediate sample value; applying the second stage ALF in reconstructed sampling, wherein applying the second stage ALF comprises: determining a second class index for the reconstructed sampling; selecting a second filter from a second set of filters based on the second class index; applying the second filter to The reconstructed samples are used to determine a first sample modification value; a second sample modification value is determined based on the first intermediate sample value; and a filtered reconstructed sample is determined based on the reconstructed samples, the first sample modification value, and the second sample modification value.

According to one example, an apparatus for decoding video data includes: memory configured to store video data; one or more processors implemented in circuitry and configured to: apply reconstruction samples to reconstruction blocks A first-stage self-adjusting loop filter (ALF), wherein to apply the first-stage ALF, the one or more processors are also configured to: reconstruct a first class index of samples; selecting filters from the first set of filters; and applying filters from the first set of filters to the reconstructed samples to determine a first intermediate sample value; applying a second stage ALF to the reconstructed samples, wherein to apply the second stage ALF, The one or more processors are also configured to: determine a second class index for the reconstructed samples; select a second filter from the second set of filters based on the second class index; apply the second filter to the reconstructed samples to A first sample modification value is determined; a second sample modification value is determined based on the first intermediate sample value; and a filtered reconstructed sample is determined based on the reconstructed sample, the first sample modification value, and the second sample modification value.

According to one example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to: apply first-stage self-adjustment to reconstructed samples of reconstructed blocks an in-loop filter (ALF), wherein to apply the first stage ALF, the one or more processors are also configured to: determine a first class index for reconstructing samples; and, applying filters from the first set of filters to the reconstructed samples to determine a first intermediate sample value; applying a second-stage ALF to the reconstructed samples, wherein in order to apply the second-stage ALF, one or more The processor is also configured to: determine a second class index for the reconstructed samples; select a second filter from the second set of filters based on the second class index; apply the second filter to the reconstructed samples to determine the first sample a modification value; determining a second sample modification value based on the first intermediate sample value; and determining a filtered reconstructed sample based on the reconstructed sample, the first sample modification value, and the second sample modification value.

The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects and advantages will be apparent from the description, drawings and claims.

Video coding (eg, video encoding and/or video decoding) typically involves coded blocks of video data from the same picture (ie, in-frame prediction) or coded blocks of video in different pictures (ie, in-frame prediction). That is, inter-frame prediction) to predict blocks of video data. In some cases, the video transcoder also computes residual data by comparing the predicted block to the original block. Therefore, residual data represents the difference between the predicted block and the original block. The video transcoder transforms and quantizes the residual data, and signals the transformed and quantized residual data in the encoded bitstream. The video decoder adds residual data to the prediction block to produce a reconstructed video block that better matches the original video block than the prediction block alone. To further improve the quality of the decoded video, the video decoder may perform one or more filtering operations on the reconstructed video blocks. Examples of these filtering operations include deblocking filtering, sample self-adjusting offset (SAO) filtering, and self-adjusting loop filtering (ALF). The parameters used for these filtering operations may be determined by the video transcoder and signaled explicitly in the encoded video bitstream, or may be determined implicitly by the video decoder without requiring a signal in the encoded video bitstream. Explicitly signaled in the bitstream.

The subject matter describes techniques associated with filtering reconstructed video data in video encoding and/or video decoding procedures, and more particularly, the subject matter describes techniques related to ALF. However, the described techniques can potentially be applied to other filtering schemes as well.

In accordance with the techniques of this disclosure, a video decoder may be configured to perform two-stage self-adjusting loop filtering. For the first stage ALF, the video decoder may be configured to determine activity and orientation values for the filtered reconstructed samples. The activity value may generally indicate a change in the sample value in the vicinity of the sample around the filtered sample. The direction value can generally indicate the direction in which the sample value changes, such as whether the sample value increases horizontally, vertically, at 45 degrees, at 135 degrees, or not at all. Based on the activity value and the direction value, the video decoder can determine the class index of the sample being filtered. Class indices may be associated with filters from the first set of filters, where each filter is defined by the shape and set of coefficient values. The video decoder can decide the first set of filters and which classifiers map to which filters based on the syntax included in the bitstream. The video decoder may apply a filter to the reconstructed samples to determine the first intermediate sample value.

For the second stage ALF, the video decoder determines a first sample modification value by applying a second filter to the reconstructed samples, and determines a second sample modification value based on the first intermediate sample value. The video decoder then determines the filtered reconstructed samples based on the reconstructed samples, the first sample modification value and the second sample modification value. As described herein, by applying a first-stage ALF to the reconstructed samples of the reconstruction block and a second-stage ALF to the reconstructed samples, the techniques disclosed in this case can do better than single-stage ALF by taking into account the local characteristics of the video data, to improve the overall quality of the decoded video data.

FIG. 1 is a block diagram illustrating an example video encoding and decoding system 100 that may implement the techniques disclosed herein. In general, the technology of the subject matter involves the decoding (encoding and/or decoding) of video data. Generally, video data includes any data used to process video. Thus, video data may include original unencoded video, encoded video, decoded (eg, reconstructed) video, and video metadata (eg, signaling data).

As shown in FIG. 1 , in this example, system 100 includes source device 102 that provides encoded video material to be decoded and displayed by destination device 116 . Specifically, source device 102 provides video material to destination device 116 via computer-readable medium 110 . Source device 102 and destination device 116 may include any of a wide variety of devices, including desktop computers, notebook (ie, laptop) computers, mobile devices, tablet computers, set-top boxes, such as smart Telephone handsets such as telephones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication, and thus may be referred to as wireless communication devices.

In the example of FIG. 1 , source device 102 includes video source 104 , memory 106 , video transcoder 200 , and output interface 108 . Destination device 116 includes input interface 122 , video decoder 300 , memory 120 , and display device 118 . In accordance with the teachings, video transcoder 200 of source device 102 and video decoder 300 of destination device 116 may be configured to apply techniques for self-adjusting loop filtering using first and second stages in the manner described herein. Thus, source device 102 represents an instance of a video encoding device, and destination device 116 represents an instance of a video decoding device. In other instances, the source and destination devices may include other components or arrangements. For example, source device 102 may receive video material from an external video source, such as an external camera. Likewise, the destination device 116 may interface with an external display device instead of including an integrated display device.

The system 100 shown in FIG. 1 is but one example. In general, any digital video encoding and/or decoding device can perform the techniques for self-adjusting loop filtering described herein. Source device 102 and destination device 116 are merely examples of such decoding devices, where source device 102 generates decoded video material for transmission to destination device 116 . The context of this case refers to a "decoding" device as a device that performs decoding (eg, encoding and/or decoding) of data. Thus, the video transcoder 200 and the video decoder 300 respectively represent examples of decoding devices (specifically, a video transcoder and a video decoder). In some examples, source device 102 and destination device 116 may operate in a substantially symmetrical manner, such that source device 102 and destination device 116 each include components for video encoding and decoding. Thus, system 100 can support one-way or two-way video transmission between source device 102 and destination device 116, eg, for video data streaming, video playback, video broadcasting, or video telephony.

Typically, video source 104 represents a source of video data (ie, raw undecoded video data) and provides a sequential series of pictures (also referred to as "frames") of video data to video transcoder 200, Video transcoder 200 encodes data for pictures. The video source 104 of the source device 102 may include a video capture device, such as a video camera, a video archive unit containing previously captured raw video, and/or a video feed interface for receiving video from a video content provider. As a further alternative, the video source 104 may generate computer graphics-based material as the source video, or a combination of real-time video, archived video, and computer-generated video. In each case, video transcoder 200 may encode captured, pre-fetched or computer-generated video data. Video transcoder 200 may rearrange the pictures from the received order (sometimes referred to as "display order") to the decoding order for decoding. Video transcoder 200 may generate a bitstream including encoded video data. The source device 102 may then output the encoded video data to the computer-readable medium 110 via the output interface 108 for reception and/or retrieval by, for example, the input interface 122 of the destination device 116 .

Memory 106 of source device 102 and memory 120 of destination device 116 represent general purpose memory. In some examples, memories 106 , 120 may store raw video data, eg, raw video from video source 104 and raw decoded video data from video decoder 300 . Additionally or alternatively, the memories 106, 120 may store software instructions executable by, for example, the video transcoder 200 and the video decoder 300, respectively. Although memory 106 and memory 120 are shown separate from video transcoder 200 and video decoder 300 in this example, it should be understood that video transcoder 200 and video decoder 300 may also include Internal memory for a functionally similar or equivalent purpose. In addition, the memories 106 , 120 may store, for example, encoded video data output from the video transcoder 200 and input to the video decoder 300 . In some examples, portions of memory 106, 120 may be allocated as one or more video buffers, eg, to store raw decoded and/or encoded video data.

Computer-readable media 110 may represent any type of media or device capable of transporting encoded video material from source device 102 to destination device 116 . In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to send encoded video data directly to destination device 116 in real time, such as via a radio frequency network or a computer-based network. The output interface 108 can modulate the transmission signal including the encoded video data, and the input interface 122 can demodulate the received transmission signal according to a communication standard, such as a wireless communication protocol. Communication media may include any wireless or wired communication media, eg, the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. Communication media may include routers, switches, base stations, or any other device that may be useful for facilitating communication from source device 102 to destination device 116 .

In some examples, source device 102 may output encoded data from output interface 108 to storage device 112 . Similarly, destination device 116 may access encoded data from storage device 112 via input interface 122 . Storage device 112 may include any of a variety of distributed or locally accessible data storage media, such as hard disks, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or Any other suitable digital storage medium for storing encoded video data.

In some examples, source device 102 may output the encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102 . The destination device 116 may access the stored video data from the file server 114 via data streaming or downloading.

File server 114 may be any type of server device capable of storing encoded video data and sending the encoded video data to destination device 116 . File server 114 may represent a web server (eg, for a website), a server configured to provide file transfer protocol services, such as file transfer protocol (FTP) or file transfer over one-way transfer (FLUTE) protocol , Content Delivery Network (CDN) devices, Hypertext Transfer Protocol (HTTP) servers, Multimedia Broadcast Multicast Service (MBMS) or Enhanced MBMS (eMBMS) servers, and/or Network Additional Storage (NAS) devices. Additionally or alternatively, file server 114 may implement one or more HTTP streaming protocols, eg, Dynamic Self-Adjusting Streaming over HTTP (DASH), HTTP Real-Time Streaming (HLS), Real-Time Data Streaming Protocol (RTSP) ), HTTP dynamic streaming, etc.

The destination device 116 may access the encoded video data from the file server 114 via any standard data connection, including an Internet connection. This may include wireless channels (eg, Wi-Fi connections), wired connections (eg, digital subscriber line (DSL), cable modems, etc.) suitable for accessing encoded video data stored on file server 114 ), or a combination of the two. Input interface 122 may be configured to operate in accordance with any one or more of the various protocols discussed above for retrieving or receiving media data from file server 114 or other such protocols for retrieving media data.

Output interface 108 and input interface 122 may represent wireless transmitters/receivers, modems, wired networking units (eg, Ethernet cards), wireless communication components operating in accordance with any of the various IEEE 802.11 standards, or other solid parts. In examples in which output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured in accordance with cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), LTE-Advanced, 5G, etc.) to transmit data (such as encoded video data). In some instances where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured in accordance with other wireless standards such as the IEEE 802.11 specification, the IEEE 802.15 specification (eg, ZigBee™), the Bluetooth™ standard, and the like ) to transmit data (such as encoded video data). In some examples, source device 102 and/or destination device 116 may include respective system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing the functions assigned to video transcoder 200 and/or output interface 108, and destination device 116 may include an SoC device for executing functions assigned to video decoder 300 and/or input interface 122-capable SoC devices.

The techniques described in this case can be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (such as HTTP-based dynamic self-adjusting data streaming (DASH)), encoding digital video onto data storage media, decoding digital video stored on data storage media, or other applications.

The input interface 122 of the destination device 116 receives the encoded video bitstream from the computer-readable medium 110 (eg, communication medium, storage device 112, file server 114, etc.). Encoded video bitstream 110 may include signaling information such as syntax elements defined by video transcoder 200 (which are also used by video decoder 300): Values of properties and/or processing of coding units (eg, slices, pictures, groups of pictures, sequences, etc.). Display device 118 displays the decoded picture of the decoded video material to the user. Display device 118 may represent any of a variety of display devices, such as a CRT, a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

Although not shown in FIG. 1, in some examples, video transcoder 200 and video decoder 300 may each be integrated with an audio encoder and/or audio decoder, and may include appropriate MUX-DEMUX units or other Hardware and/or software to process multiplexed streams including both audio and video in the common data stream. MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol or other protocols such as User Data Packet Protocol (UDP), if applicable.

Video transcoder 200 and video decoder 300 may each be implemented as any of a variety of suitable encoder and/or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), individual logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented in part in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware, using one or more processors, to carry out the invention content technology. Each of the video transcoder 200 and the video decoder 300 may be included in one or more encoders or decoders, either of which may be integrated into a combined encoding in a corresponding device part of the encoder/decoder (CODEC). Devices including video transcoder 200 and/or video decoder 300 may include integrated circuits, microprocessors, and/or wireless communication devices such as cellular telephones.

Video transcoder 200 and video decoder 300 may be based on video coding standards such as ITU-T H.265 (also known as High Efficiency Video Coding (HEVC) standard) or extensions thereto such as multi-view and/or or Scalable Video Decoding Extension)) to operate. Alternatively, video transcoder 200 and video decoder 300 may operate according to other proprietary or industry standards (eg, ITU-T H.266, also known as Versatile Video Coding (VVC)). The draft of the VVC standard was presented at the first meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 via teleconference from 22 June to 1 July 2020 Session 18, JVET-S2001-v17, described in "Versatile Video Coding (Draft 10)" by Bross et al. (hereinafter referred to as "VVC Draft 10"). However, the techniques disclosed herein are not limited to any particular coding standard, and it is expressly contemplated that the techniques described herein may be used in conjunction with subsequent standards for VVC.

In general, video transcoder 200 and video decoder 300 may perform block-based coding of pictures. The term "block" generally refers to a structure that includes data to be processed (eg, to be encoded, decoded, or otherwise used in an encoding and/or decoding process). For example, a block may include a two-dimensional matrix of samples of luma and/or chroma data. Generally, the video transcoder 200 and the video decoder 300 can decode video data represented in YUV (eg, Y, Cb, Cr) format. That is, instead of decoding red, green, and blue (RGB) data for sampling of a picture, video transcoder 200 and video decoder 300 may decode luma and chroma components, where the chroma components Both red hue and blue hue chrominance components may be included. In some examples, video transcoder 200 converts the received RGB-formatted data to a YUV representation before encoding, and video decoder 300 converts the YUV representation to RGB format. Optionally, preprocessing and postprocessing units (not shown) may perform these transformations.

The subject matter may generally relate to the decoding (eg, encoding and decoding) of a picture to include procedures for encoding or decoding the material of the picture. Similarly, the subject matter may relate to the coding of blocks of pictures to include procedures for encoding or decoding (eg, prediction and/or residual coding) data for the blocks. An encoded video bitstream typically includes a series of values for syntax elements that represent coding decisions (eg, coding modes) and partition the picture into blocks. Accordingly, references to coding a picture or block should generally be understood as coding the values of syntax elements used to form the picture or block.

HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder, such as video transcoder 200, partitions coding tree units (CTUs) into CUs according to a quad-tree structure. That is, the video decoder partitions CTUs and CUs into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node," and a CU of such a leaf node may include one or more PUs and/or one or more TUs. Video decoders can further partition PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents the partitioning of TUs. In HEVC, PU stands for inter-frame prediction data, and TU stands for residual data. An intra-frame predicted CU includes intra-frame prediction information, such as an intra-frame mode indication.

As another example, video transcoder 200 and video decoder 300 may be configured to operate according to VVC. According to VVC, a video decoder, such as video transcoder 200, partitions a picture into a plurality of coding tree units (CTUs). The video transcoder 200 may partition the CTUs according to a tree structure, such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure removes the concept of various partition types, such as the partition between CUs, PUs and TUs of HEVC. The QTBT structure includes two levels: a first level partitioned according to quadtree partitioning, and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

In an MTT partition structure, blocks may be partitioned using quadtree (QT) partitions, binary tree (BT) partitions, and one or more types of ternary tree (TT) (also known as triple tree (TT)) partitions segmentation. A ternary tree or ternary tree partition is a partition in which a block is divided into three sub-blocks. In some examples, ternary tree or ternary tree partitioning divides the block into three sub-blocks without dividing the original block via the center. Segmentation types (eg, QT, BT, and TT) in MTT can be symmetric or asymmetric.

In some examples, video transcoder 200 and video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, video transcoder 200 and video decoder The generator 300 may use two or more QTBT or MTT structures, such as one QTBT/MTT structure for the luma component and another QTBT/MTT structure for the two chroma components (or one for the corresponding chroma components). two QTBT/MTT structures).

Video transcoder 200 and video decoder 300 may be configured to use per-HEVC quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures. For explanatory purposes, a description of the techniques of the present case is provided with respect to QTBT segmentation. It should be understood, however, that the techniques of this disclosure may also be applied to video decoders configured to use quad-tree partitioning, or other types of partitioning as well.

In some examples, a CTU includes a coding tree block (CTB) of luma samples, two corresponding CTBs of chroma samples of a picture with three sample arrays, or a black and white picture or using three separate color planes and for The CTB of the sample of the picture that decodes the syntax structure that encodes the sample. A CTB may be an NxN block of samples for some N value, such that the partitioning of a component into multiple CTBs is a partition. A component is an array or a single sample from one of the three arrays (luminance and two chrominance) that make up a picture in 4:2:0, 4:2:2, or 4:4:4 color format, or An array or individual samples of an array that make up a picture in black and white format. In some examples, a coding block is an MxN block of samples for some values of M and N, such that the partitioning of one CTB into multiple coding blocks is a partition.

Blocks (eg, CTUs or CUs) may be grouped in a picture in various ways. As an example, a brick may represent a rectangular area of a CTU row within a particular tile in the picture. A tile can be a rectangular area of a CTU within a specific tile column and a specific tile row in the picture. A tile column represents a rectangular area of a CTU that has a height equal to the height of the picture and a width specified by a syntax element (eg, such as in a picture parameter set). A tile row represents a rectangular area of a CTU that has a height specified by a syntax element (eg, such as in a picture parameter set) and a width equal to the width of the picture.

In some instances, a tile can be divided into tiles, each tile can include one or more CTU rows within the tile. Tiles that are not divided into bricks can also be called bricks. However, bricks that are a true subset of tiles may not be called tiles.

The bricks in the picture can also be arranged in slices. A slice can be an integer number of bricks of a picture, which can be uniquely contained in a single Network Abstraction Layer (NAL) unit. In some instances, a slice includes multiple complete tiles or a contiguous sequence of complete bricks that includes only one tile.

This document may use "NxN" and "N by N" interchangeably to represent the sample size of a block (such as a CU or other video block) in vertical and horizontal dimension, eg, 16x16 samples or 16 by 16 samples. Typically, a 16x16 CU will have 16 samples vertically (y=16) and 16 samples horizontally (x=16). Likewise, an NxN CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in the CU can be arranged in rows and columns. Furthermore, a CU does not necessarily need to have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include NxM samples, where M is not necessarily equal to N.

Video transcoder 200 encodes video data representing prediction and/or residual information and other information for the CU. Prediction information indicates how the CU is to be predicted in order to form a prediction block for the CU. Residual information typically represents the sample-by-sample differences between the samples of the CU prior to encoding and the prediction block.

To predict a CU, video transcoder 200 may typically form prediction blocks for the CU through inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting a CU from data from previously coded pictures, while intra-frame prediction generally refers to predicting a CU from previously coded data of the same picture. To perform inter-frame prediction, video transcoder 200 may use one or more motion vectors to generate prediction blocks. Video transcoder 200 may typically perform a motion search to identify reference blocks that closely match the CU, eg, the difference aspect between the CU and the reference block. Video transcoder 200 may use sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute differences (MAD), mean square deviations (MSD), or other such difference calculations to calculate a difference metric to determine the reference block Whether it closely matches the current CU. In some examples, video transcoder 200 may predict the current CU using uni-directional prediction or bi-directional prediction.

Some examples of VVC also provide an affine motion compensation mode, which can be thought of as an inter-frame prediction mode. In affine motion compensation mode, video transcoder 200 may determine two or more motion vectors that represent non-translational motion, such as zoom-in or zoom-out, rotation, perspective motion, or other irregular motion types.

To perform intra-frame prediction, video transcoder 200 may select an intra-frame prediction mode to generate prediction blocks. Some examples of VVC provide sixty-seven intra-frame prediction modes, including various directional modes, as well as planar and DC modes. Typically, video transcoder 200 selects an intra-frame prediction mode that describes the adjacent samples of the current block from which samples of the current block (eg, a block of a CU) are to be predicted. Assuming that video transcoder 200 decodes CTUs and CUs in raster scan order (left-to-right, top-to-bottom), such samples may typically be above the current block in the same picture as the current block, top left or left.

Video transcoder 200 encodes data representing the prediction mode for the current block. For example, for an inter-frame prediction mode, video transcoder 200 may encode data indicating which of the various available inter-frame prediction modes to use and motion information for the corresponding mode. For uni-directional or bi-directional inter-frame prediction, for example, video transcoder 200 may use Advanced Motion Vector Prediction (AMVP) or merge mode to encode motion vectors. Video transcoder 200 may use a similar mode to encode motion vectors for affine motion compensation mode.

After prediction, such as intra-frame prediction or inter-frame prediction for a block, video transcoder 200 may compute residual data for the block. Residual data, such as a residual block, represent the sample-by-sample differences between the block and the prediction block for that block, which was formed using the corresponding prediction mode. Video transcoder 200 may apply one or more transforms to the residual block to produce transformed data in the transform domain rather than the sample domain. For example, video transcoder 200 may apply a discrete cosine transform (DCT), integer transform, wavelet transform, or conceptually similar transform to the residual video data. Additionally, the video transcoder 200 may apply a secondary transform, such as Mode Dependent Non-Separable Secondary Transform (MDNSST), Signal Dependent Transform, Karhunen-Loeve Transform (KLT), etc., after the first transform. Video transcoder 200 generates transform coefficients after applying one or more transforms.

As previously described, after any transform to generate transform coefficients, video transcoder 200 may perform quantization of the transform coefficients. Quantization generally represents a procedure in which transform coefficients are quantized to possibly reduce the amount of data used to represent these transform coefficients, thereby providing further compression. By performing a quantization procedure, video transcoder 200 may reduce the bit depth associated with some or all transform coefficients. For example, video transcoder 200 may round down an n- bit value to an m -bit value during quantization, where n is greater than m . In some examples, to perform quantization, video transcoder 200 may perform a bitwise right shift of the value to be quantized.

After quantization, video transcoder 200 may scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. A scan can be designed to place higher energy (and therefore lower frequency) transform coefficients at the front of the vector, and lower energy (and therefore higher frequency) transform coefficients at the back of the vector. In some examples, video transcoder 200 may scan the quantized transform coefficients using a predefined scan order to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, the video transcoder 200 may perform a self-adjusting scan. After scanning the quantized transform coefficients to form a one-dimensional vector, video transcoder 200 may entropy encode the one-dimensional vector, eg, according to context self-adjusting binary arithmetic coding (CABAC). Video transcoder 200 may also entropy encode the values of syntax elements used to describe metadata associated with the encoded video data for use by video decoder 300 in decoding the video data.

To perform CABAC, video transcoder 200 may assign contexts within a context model to symbols to be transmitted. The context may relate, for example, to whether adjacent values of a symbol are zero-valued. Probabilistic decisions may be based on the context assigned to the symbol.

Video transcoder 200 may also generate syntax data (such as block-based syntax data, picture-based syntax data, and sequence-based syntax data) for video decoder 300, eg, in picture headers, block headers, slice headers, Or other syntax data (such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS) or Video Parameter Set (VPS)). Likewise, the video decoder 300 can decode such syntax data to determine how to decode the corresponding video data.

In this manner, video transcoder 200 may generate a bitstream that includes encoded video data, eg, describing partitioning of a picture into blocks (eg, CUs) and predictions and/or residuals for the blocks Syntax elements of information. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

Typically, video decoder 300 performs the reverse of the process performed by video transcoder 200 to decode the encoded video data of the bitstream. For example, video decoder 300 may use CABAC to decode the values of syntax elements for a bitstream in a substantially similar but reversed manner as the CABAC encoding procedure of video transcoder 200. The syntax elements may define partitioning information for partitioning a picture into CTUs, and partitioning each CTU according to a corresponding partitioning structure, such as a QTBT structure, to define the CUs of the CTU. Syntax elements may also define prediction and residual information for blocks of video data (eg, CUs).

Residual information may be represented by, for example, quantized transform coefficients. Video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of a block to reproduce a residual block for the block. Video decoder 300 uses the signaled prediction mode (intra-frame prediction or inter-frame prediction) and associated prediction information (eg, motion information for inter-frame prediction) to form a prediction block for the block. Video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. Video decoder 300 may perform additional processing, such as performing deblocking procedures to reduce visual artifacts along block boundaries.

The content of the case may generally involve the "signaling" of certain information (such as grammatical elements). The term "signaling" may generally refer to the transmission of values for syntax elements and/or other data for decoding encoded video data. That is, video transcoder 200 may signal values for syntax elements in the bitstream. Typically, signaling means generating values in the bitstream. As previously described, source device 102 may stream the bitstream to the destination substantially instantaneously or not (such as may occur when syntax elements are stored to storage 112 for later retrieval by destination device 116) device 116 .

In video coding, such as in the H.266/VVC standard, ALF is used to minimize the mean squared error between filtered samples and original samples. For example, the input samples for the ALF may be the output samples of the SAO. ALF output samples can be stored in the decoded picture buffer (DPB) or output as visible pictures. The ALF filter shapes used in the Joint Exploration Model (JEM) software are 5x5, 7x7 and 9x9 diamonds. Filter shapes can be selected and signaled at the picture level in JEM. In order to achieve a better compromise between decoding efficiency and filter complexity, in VVC, only 7x7 diamonds and 5x5 diamonds are supported for the luma and chroma components, respectively.

以7位小數精度來表示。

的絕對值是經由使用第0階指數哥倫布（Exp-Golomb）碼後面跟著用於非零係數的符號位元而被譯碼的。在圖2A和圖2B中，每個方塊對應一個亮度或色度取樣，並且中心方塊對應當前待濾波取樣。為了減少發送係數的管理負擔和乘法的次數，圖2A和圖2B中的濾波器形狀是點對稱的。另外，如等式（1）所示，所有濾波器係數之和被設置為128，其是具有7位小數精度的1.0的固定點表示。   

（1） 2A illustrates an example filter 140, which is a 7x7 diamond filter. FIG. 2B illustrates an example filter 142, which is a 5x5 diamond filter. In each of filters 140 and 142, the integer coefficients

Expressed with 7 decimal places of precision.

The absolute value of is decoded using a 0th order Exp-Golomb code followed by a sign bit for the non-zero coefficients. In Figures 2A and 2B, each block corresponds to a luma or chroma sample, and the center block corresponds to the current sample to be filtered. In order to reduce the management burden of sending coefficients and the number of multiplications, the filter shapes in Figures 2A and 2B are point-symmetric. Additionally, as shown in equation (1), the sum of all filter coefficients is set to 128, which is a fixed point representation of 1.0 with 7 decimal places of precision.

(1)

In equation (1), N is the number of coefficients and is equal to 13 and 7 for 7x7 and 5x5 filter shapes, respectively.

   （2） 其中R（x,y）是SAO之後的取樣值。 In VVC, nonlinearity is introduced into the ALF. When the difference between the adjacent sample value and the current sample value to be filtered is too large, a simple trimming function is applied to reduce the influence of the adjacent sample value. To filter the samples, ALF can be performed as:

(2) where R(x,y) is the sampled value after SAO.

   （3） 其中j等於0或1，並且

是第i個係數

的濾波器分接點位置偏移。 Use the trim function to define the nonlinear function as:

(3) where j is equal to 0 or 1, and

is the ith coefficient

The filter tap position offset of .

的修剪參數

是由修剪索引

決定的。BD是內部位元深度。   

（4） In VVC version 1, as shown in equation (4), for coefficients

pruning parameters for

is indexed by pruning

decided. BD is the internal bit depth.

(4)

可以是0、1、2或3，並且經由使用2位元固定長度碼用訊號發送。為了簡化修剪操作，如在等式（4）中，修剪參數

的值可以被限制為僅為2的冪。因此，可以應用逐位元邏輯操作，作為修剪操作。 For the filter, the number of signal coefficients and the number of signal trim indices are both N-1. Each coefficient is restricted to the range [-128, 127], which is equivalent to [-1.0, 1.0] with 7 decimal places of precision. per pruned index

Can be 0, 1, 2, or 3, and is signaled using a 2-bit fixed-length code. To simplify the pruning operation, as in equation (4), the pruning parameter

The value of can be restricted to only powers of 2. Therefore, a bitwise logical operation can be applied as a trimming operation.

Video transcoder 200 and video decoder 300 may be configured to perform sub-block level filter self-adjustment. In VVC version 1, ALF follows the same luminance classification framework as ALF in JEM-7.0. To obtain a better compromise between coding efficiency and computational complexity, the block size for classification may be increased from 2x2 samples to 4x4 samples. In order to determine the class index of the 4x4 block, a surrounding window with 8x8 luma samples is used to derive orientation and motion information. In this 8x8 luma sample window, four gradient values for every other sample are first calculated, as shown in Figure 3. Figure 3 illustrates the sub-sampled Laplacian values of the 4x4 sub-block 150 for ALF classification. Calculates the gradient value for the samples marked with dots. The gradient values of other samples are set to 0.

（5） Figure 4 illustrates four gradient values for each sample with coordinates (k,l). The dots represent the samples for which the gradient is being calculated. Block 160 illustrates the horizontal gradient (H), and block 162 illustrates the vertical gradient (V). Block 164 illustrates a 135 degree gradient ( D1 ), and block 166 illustrates a 45 degree gradient ( D2 ). H, V, D1 and D2 are derived as follows:

(5)

（6） Variable i and variable j may refer to the coordinates of the upper left sample in the 4x4 block. The calculated sum of the horizontal gradient g _H , the vertical gradient g _V , the 135-degree gradient g _D1 and the 45-degree gradient g _D2 is calculated as follows:

(6)

表示的水平和垂直梯度的最大值和最小值之比，以及用

表示的兩個對角線梯度的最大值和最小值之比，被計算如等式（7）中所示。   

（7） use

represents the ratio of the maximum and minimum values of the horizontal and vertical gradients, and uses

The ratio of the maximum and minimum values of the two diagonal gradients, represented, is calculated as shown in equation (7).

(7)

和

彼此與兩個閾值

和

進行比較，以推導出方向性D： 步驟1：若

且

，則D被設置為0（紋理），否則繼續步驟2。 步驟2：若

，則繼續步驟3，否則繼續步驟4。 步驟3：若

，則D被設置為1（弱對角線），否則D被設置為2（強對角線）。 步驟4：若

，則D被設置為3（弱水平/垂直），否則D被設置為4（強水平/垂直）。 Subsequently, the

and

each other with two thresholds

and

Compare to derive the directionality D: Step 1: If

and

, then D is set to 0 (texture), otherwise proceed to step 2. Step 2: If

, go to step 3, otherwise go to step 4. Step 3: If

, then D is set to 1 (weak diagonal), otherwise D is set to 2 (strong diagonal). Step 4: If

, then D is set to 3 (weak horizontal/vertical), otherwise D is set to 4 (strong horizontal/vertical).

（8） The activity value A is calculated as follows:

(8)

的值的類別。5x5網格170表示25個類別，並且5x5網格170的每個框內的數字表示從0到6的合併類別。每個類別，即5x5網格170之每一者方塊，可以具有從0到24的閉區間的索引。A被映射到0到4的閉區間，並且量化值被表示為 Â。因此，每個4x4塊被歸類到25個類別之一，如下所示：   

（9） Figure 5 illustrates an example of merging 25 luminance categories into 7 merged categories (0 to 6), where each square represents a

The category of the value. The 5x5 grid 170 represents 25 categories, and the numbers within each box of the 5x5 grid 170 represent the combined categories from 0 to 6. Each category, ie each square of the 5x5 grid 170, may have an index of a closed interval from 0 to 24. A is mapped to a closed interval from 0 to 4, and the quantized value is denoted as  . Therefore, each 4x4 block is classified into one of 25 categories as follows:

(9)

A luminance filter bank contains 25 filters. However, in order to reduce the number of bits required to represent filter coefficients while maintaining coding efficiency, different classes can be combined and the combined classes use the same filter. Consolidate tables are signaled. In the merge table, for example, the filter index for each class is signaled using a fixed length code. Signaling is used in the example filter banks of the luma filters of Figures 5 and 7. For each class, the filter index (in this example, from 0 to 6) is signaled in ALF_APS.

 並且

無變換

對角線翻轉

垂直翻轉

右旋轉 After the filter is determined from the luma filter bank based on the class index C of the 4x4 block and the merging table, before filtering the samples of the 4x4 block, a geometric transformation can be applied to the filter according to the gradient values calculated for the 4x4 block, as shown in the table shown in 1. Table 2 Geometric transformations based on gradient values gradient value transform

and

no transformation

Diagonal flip

flip vertically

rotate right

FIG. 6 illustrates an example of a geometric transformation of filter 140 in FIG. 2A. As shown in FIG. 6 , filter 180 corresponds to a diagonal flip of filter 140 . Filter 182 corresponds to a vertical flip of filter 140 , and filter 184 corresponds to a right rotation of filter 140 .

Video transcoder 200 and video decoder 300 may be configured to perform coding tree block-level self-adjustment. In JEM-7.0, only one luma filter bank is applied to all luma CTBs of a slice, and only one chroma filter is applied to all chroma CTBs of a slice. However, there are two potential drawbacks. First, when there is a certain amount of difference in statistics between CTBs, using the same filter or filter bank for all CTBs of a color component may limit the decoding efficiency of ALF, especially for high-resolution sequences and mixed content video sequence. Second, when pushing a filter for a slice, the filter cannot be computed until statistics for the entire slice are collected. This multi-pass decoding is not friendly to low-latency applications. To address this problem, one solution is to use statistics from previously coded slices. However, this may result in some performance loss.

Besides luminance 4x4 block-level filter self-adjustment, VVC also supports CTB-level filter self-adjustment. Within a slice, different luma CTBs are allowed to use different luma filter banks, and different chroma CTBs can use different chroma filters. CTBs with similar statistics can use the same filter. The CTB stage filter self-adjustment improves decoding efficiency, especially for low-latency applications. Furthermore, VVC version 1 allows filters from previously coded pictures to be used for CTBs. The temporal filter reuse mechanism can reduce the management burden of filter coefficient signaling. In VVC version 1, up to 7 signal luminance filter banks and 8 signal chrominance filters can be applied to a slice. When there are no filters to signal, one of 16 fixed filter banks can be applied to the luma CTB. When ALF is enabled, the filter bank index of the fixed filter bank or the signal luminance filter bank is signaled for the luma CTB. For CTB, the filter index of the signal chroma filter is signaled. By using filters signaled from previously decoded pictures and fixed filters, 3 CTU levels can be decided on/off by using only statistics for the current CTU when encoding the current CTU in low latency applications Flags and filter/filterbank index. Thus, the coded bitstream for each CTU can be generated on the fly without waiting for statistics for the entire picture to be available.

Video transcoder 200 and video decoder 300 may be configured to perform techniques for line buffer reduction. As shown in FIGS. 2A and 2B , in the vertical direction, the filter shape has 7 tap points for the luma component and 5 tap points for the chroma component, respectively. Therefore, in VVC Test Model 2.0 (VTM-2.0), when decoding a row of CTUs, the 7 luma lines and 4 chrominance lines of the upper CTU row must be stored in the in the line buffer of the ALF. However, additional line buffers require large die area, especially for high-definition (HD) and ultra-high-definition (UHD) video sequences.

To make the ALF hardware friendly (eg, by reducing line buffer requirements), the concept of virtual boundaries (VBs) can be applied to remove all line buffer management burdens for the ALF. Considering the deblocking filter and SAO filter in VVC version 1, the position of the VB is 4 luma samples and 2 chroma samples above the horizontal CTU boundary. When filtering one sample on one side of the VB, the samples on the other side of the VB cannot be utilized and modified filtering with symmetric sample padding can be applied.

7A-7C illustrate an example of symmetric sample padding for luma ALF filtering at ALF VB. In the example of Figures 7A-7C, the center square of filter 190 is the position of the current sample to be filtered, and the thick line is the position of VB (VB 192). In Figures 7A-7C, filter tap locations are populated with dashed lines. FIG. 7A illustrates an example in which one of the filter taps of filter 190 is positioned above or below VB 192 . In this example, one filter tap location is populated. FIG. 7B illustrates an example in which the 4 filter taps of filter 190 are positioned above or below VB 192 . In this example, 4 filter tap locations are populated.

   （10） However, when the samples are on the nearest line on each side of the VB 192, as shown in Figure 7C, the 2D filter is equivalent to a horizontal filter. This may introduce visual artifacts. To solve this problem, the filter strength can be compensated when the current sample to be filtered is on the nearest row on each side of the VB, as shown in equation (10). Comparing Equation (10) with Equation (2), it can be seen that there is an extra shift of 3 bits to the right.

(10)

The classification of 4x4 blocks can also be modified when VB processing is applied. Gradients and sampling on the other side of the VB may not be used when computing the class index of the 4x4 block on one side of the VB, as shown in FIG. 8 .

（11） Figure 8 illustrates an example of ALF 4x4 sub-block classification at ALF VB. When calculating gradient values for samples adjacent to the VB, the samples on the other side of the VB cannot be utilized. Therefore, the boundary samples on the current side are repeatedly expanded, as shown in FIG. 8 . That is, the boundary samples on the current side of the VB are mirrored to the other side of the VB. As the number of available gradient values decreases, the activity derivation in equation (8) is rescaled to:

(11)

Video transcoder 200 and video decoder 300 may be configured to perform filter coefficient signaling. In VVC version 1, the ALF coefficients are signaled in the ALF Self-Tuning Parameter Set (APS). An APS can contain a set of luma filters, with up to 25 filters, up to 8 chroma filters, and up to 8 cross-component ALF (CC-ALF) filters. Each set of luma filters supports applying ALF to the luma 25 class. In VVC version 1, up to 8 ALF_APS are supported.

Table 2 below illustrates an example syntax signaling table for signaling filter coefficients in accordance with the techniques of the present disclosure. Table 2 alf_data(){ Descriptor alf_luma_filter_signal_flag u(1) if(aps_chroma_present_flag){ alf_chroma_filter_signal_flag u(1) alf_cc_cb_filter_signal_flag u(1) alf_cc_cr_filter_signal_flag u(1) } if(alf_luma_filter_signal_flag){ alf_luma_clip_flag u(1) alf_luma_num_filters_signalled_minus1 ue(v) if(alf_luma_num_filters_signalled_minus1>0) for( filtIdx = 0; filtIdx <NumAlfFilters; filtIdx++ ) alf_luma_coeff_delta_idx[filtIdx] u(v) for( sfIdx = 0; sfIdx <= alf_luma_num_filters_signalled_minus1; sfIdx++ ) for( j = 0; j <12; j++ ){ alf_luma_coeff_abs[ sfIdx ][ j ] ue(v) if(alf_luma_coeff_abs[sfIdx][j]) alf_luma_coeff_sign[ sfIdx ][ j ] u(1) } if(alf_luma_clip_flag) for( sfIdx = 0; sfIdx <= alf_luma_num_filters_signalled_minus1; sfIdx++ ) for( j = 0; j <12; j++ ) alf_luma_clip_idx[ sfIdx ][ j ] u(2) } if(alf_chroma_filter_signal_flag){ alf_chroma_clip_flag u(1) alf_chroma_num_alt_filters_minus1 ue(v) for(altIdx = 0; altIdx <= alf_chroma_num_alt_filters_minus1; altIdx++ ){ for( j = 0; j <6; j++ ){ alf_chroma_coeff_abs[altIdx][j] ue(v) if (alf_chroma_coeff_abs[altIdx][j] > 0) alf_chroma_coeff_sign[altIdx][j] u(1) } if(alf_chroma_clip_flag) for( j = 0; j <6; j++ ) alf_chroma_clip_idx[altIdx][j] u(2) } } if(alf_cc_cb_filter_signal_flag){ alf_cc_cb_filters_signalled_minus1 ue(v) for( k = 0; k < alf_cc_cb_filters_signalled_minus1 + 1; k++ ){ for( j = 0; j <7; j++ ){ alf_cc_cb_mapped_coeff_abs[ k ][ j ] u(3) if(alf_cc_cb_mapped_coeff_abs[ k ][ j ] ) alf_cc_cb_coeff_sign[ k ][ j ] u(1) } } } if(alf_cc_cr_filter_signal_flag){ alf_cc_cr_filters_signalled_minus1 ue(v) for( k = 0; k < alf_cc_cr_filters_signalled_minus1 + 1; k++ ){ for( j = 0; j <7; j++ ){ alf_cc_cr_mapped_coeff_abs[ k ][ j ] u(3) if(alf_cc_cr_mapped_coeff_abs[ k ][ j ] ) alf_cc_cr_coeff_sign[ k ][ j ] u(1) } } } }

The content of this case describes techniques that can further improve the effectiveness of ALF. For example, in VVC, when filtering a sample, only one classifier and one filter can be applied. However, this case describes techniques for applying multiple classifiers and filters to capture more local features. The content of this case describes techniques for a framework to filter samples using multiple filters and classifiers.

9 illustrates an example framework for filtering reconstructed samples of video data using multiple fixed filter banks and multiple signaled filter banks. Video transcoder 200 and video decoder 300 may be configured to implement multiple fixed filters and multiple signaled filters in accordance with the techniques of FIG. 9 . The techniques of FIGS. 10-15 will be described with respect to video decoder 300 , but the techniques may also be performed by video transcoder 200 .

In the example of Figure 9, the video decoder 300 applies the first stage ALF 400 to the reconstructed samples R(x,y) to determine the intermediate filtered signal R'. Subsequently, the video decoder 300 applies the second stage ALF 410 to R&apos; to determine the value of the filtered samples 430.

和

表示可以向取樣應用的用訊號發送的濾波器組和固定（預定義）濾波器組的數量。Rs是對ALF的輸入取樣。 F（f, i）其中i = 0…N _f-1表示第i個固定濾波器組 C（f, i）其中i = 0…N _f-1 表示針對第i個固定濾波器組的分類器。經由C（f, i），對於每個取樣，計算濾波器索引。基於濾波器索引，從固定濾波器組F（f, i）中選擇濾波器來濾波該取樣。此外，C（f, i）決定如何將幾何變換應用於係數。 F（s, i）其中i = 0…N _s-1表示第i個用訊號發送的濾波器組。 C（s, i）其中i = 0…N _s-1表示具有第i個用訊號發送的濾波器組的分類器。經由C（s, i），對於每個取樣，計算濾波器索引。基於濾波器索引，從固定濾波器組F（s, i）中選擇濾波器來對該取樣進行濾波。此外，C（s, i）決定如何對係數應用幾何變換。 The aspect of the first stage ALF 400 will now be described in more detail.

and

Indicates the number of signaled filter banks and fixed (predefined) filter banks that can be applied to the sample. Rs is the input sample to the ALF. F(f, i) where i = 0...N _f -1 denotes the ith fixed filter bank C(f, i) where i = 0...N _f -1 denotes the classifier for the ith fixed filter bank . Via C(f, i), for each sample, the filter index is calculated. Based on the filter index, a filter is selected from the fixed filter bank F(f, i) to filter the sample. Furthermore, C(f, i) decides how the geometric transformation is applied to the coefficients. F(s, i) where i = 0...N _s -1 denotes the ith signaled filter bank. C(s, i) where i = 0...N _s -1 denotes the classifier with the ith signaled filter bank. Via C(s, i), for each sample, the filter index is calculated. Based on the filter index, a filter is selected from the fixed filter bank F(s, i) to filter the sample. Furthermore, C(s, i) decides how to apply the geometric transformation to the coefficients.

（12） For reconstructed samples R(x, y), video decoder 300 may apply multiple filter banks by using the reconstructed samples themselves and adjacent samples. Given a filter bank (which can be a fixed filter bank or a signaled filter bank), a classifier can be applied. The classifier decides which filter from the filter bank can be applied and how to transpose the filter coefficients. After the first stage, for each filter bank and corresponding classifier, an intermediate filtered signal R' can be calculated, e.g.,

(12)

，

是第i個濾波器組中所選濾波器的第j個係數，Ni是係數的數量，f _i,j,k其中k=0或1是一個相鄰取樣和當前取樣的函數，例如，f _i,j,k可以被實現為VVC中的修剪函數

（13） （

是相鄰取樣相對於當前取樣的座標偏移量。 For the ith filter, where i = 0...

,

is the jth coefficient of the selected filter in the ith filter bank, Ni is the number of coefficients, f _i,j,k where k = 0 or 1 is a function of the adjacent sample and the current sample, e.g., f _i,j,k can be implemented as trim functions in VVC

(13) (

is the coordinate offset of adjacent samples relative to the current sample.

In some examples, the filter shapes of the fixed or signaled filters may be 5x5, 7x7, 9x9, 11x11, and 13x13. Figure 10A illustrates an example of filter 510, which is a 5x5 diamond filter shape, and Figure 10B illustrates an example of filter 520, which is a 7x7 diamond filter shape. Figure 10C illustrates an example of filter 530, which is a 9x9 diamond filter shape, and Figure 10D illustrates an example of filter 540, which is a 11x11 diamond filter shape. Figure 1OE illustrates an example of filter 550, which is a 13x13 diamond filter shape.

In one example, the video decoder 300 may be configured to decide the classifier C(f/s, i), activity and orientation values based on the 2-D Laplacian operator/gradient values. A classifier can be applied to each sample or block. When the classifier is applied to a block, all samples in the block have the same class index and the same transpose type. For example, _wi may represent the width of the block, _hi may represent the height of the block, and (x, y) may represent the coordinates of the upper-left corner sample of the block.

（14） For a sample with coordinates (k,l), four Laplacian (gradient) values: horizontal gradient H , vertical gradient V , 135-degree gradient Dl and 45-degree gradient D2 can be derived as

(14)

（15） 其中 a _i 和 b _i 分別是用於分類器C（f/s, i）在水平方向和垂直方向上的訊窗大小。 The video decoder 300 may be configured to derive the activity value A _i by using the vertical gradient and the horizontal gradient:

(15) where a _i and b _i are the window sizes for the classifier C(f/s, i) in the horizontal and vertical directions, respectively.

的閉區間。量化值表示為 Â _i 。 Video decoder 300 may be configured to quantize A _i to 0 to

closed interval. The quantized value is denoted as  _i .

The video decoder 300 may be configured to use the horizontal gradient H, vertical gradient V, 135 degree gradient D1 and 45 degree gradient D2 to determine the direction.

、垂直梯度

和兩個對角線梯度

和

的值，如下：

（16） The video decoder 300 may be configured to first calculate the horizontal gradient

, vertical gradient

and two diagonal gradients

and

value, as follows:

(16)

（17）

（18） 隨後，可以與VVC同樣推導方向 D _i ，其中方向的數量

= 5： 步驟1. 若

並且

,則

被設置為0（塊被歸類為「紋理」）。 步驟2. 若

，則從步驟3繼續，否則從步驟4繼續。 步驟3. 若

，則

設置為2（塊被歸類為「強的水平/垂直」），否則

設置為1（塊被歸類為「弱的水平/垂直」）。 步驟4. 若

，則

設置為4（塊被歸類為「強的對角」），否則

被設置為3（塊被歸類為「弱的對角」）。 In order to assign the directionality D _i , the video transcoder 200 and the video decoder 300 may be configured to derive maximum and minimum values of the horizontal and vertical gradients as follows:

(17)

(18) Subsequently, the direction D _i can be derived similarly to VVC, where the number of directions

= 5: Step 1. If

and

,but

is set to 0 (blocks are classified as "textures"). Step 2. If

, then continue from step 3, otherwise continue from step 4. Step 3. If

,but

Set to 2 (blocks are classified as "strong horizontal/vertical"), otherwise

Set to 1 (blocks are classified as "weak horizontal/vertical"). Step 4. If

,but

Set to 4 (blocks are classified as "strong diagonal"), otherwise

is set to 3 (blocks are classified as "weak diagonal").

In some instances, more directions may be supported.

與閾值（Th）陣列進行比較來計算水平/垂直方向（（ES _i,HV））的邊緣強度。閾值陣列的大小用b S來表示，並且該閾值是按昇冪排列的。方向的數量是

步驟1. 初始化 m = 0 並且 ES _i,HV= 0； 步驟2. 若m等於S，則停止；否則，轉到步驟3。 步驟3. 若

＞Th[m], m = m+1 並且 ES _i,HV= ES _i,HV+ 1,進入步驟2；否則，停止。 The video decoder 300 can be configured to

The edge intensities in the horizontal/vertical direction ((ES _i,HV )) were calculated by comparing with the threshold (Th) array. The size of the threshold array is denoted by b S, and the thresholds are arranged in ascending powers. The number of directions is

Step 1. Initialize m = 0 and ES _i,HV = 0; Step 2. If m equals S, stop; otherwise, go to Step 3. Step 3. If

>Th[m], m = m+1 and ES _{i, HV} = ES _{i, HV} + 1, go to step 2; otherwise, stop.

＞Th[m], m = m+1並且E _i,D= E _i,D+ 1，進入步驟2；否則，停止。 The video decoder 300 can be configured to calculate the diagonal direction (E _i,D ) as follows: Step 1. Initialize m = 0 and E _i,D = 0; Step 2. If m equals S, stop; otherwise, go to to step 3. Step 3. If

>Th[m], m = m+1 and E _{i, D} = E _{i, D} + 1, go to step 2; otherwise, stop.

An example of a threshold array is Th=[1.25, 1.5, 2, 3, 4.5, 8], and S=6.

，S _M被設置為ES _HV，ES _S被設置為ES _D；否則，ES _M被設置為ES _D，並且ES _S被設置為ES _HV若ES _S大於ES _M，則D _i被設置為0。否則，若

ES _M*（ES _M+1）/2 + ES _S；否則

ES _M*（ES _M+1）/2 + ES _S+

2 The video decoder 300 may be configured to determine the primary edge strength (ES _M ) and secondary edge strength ( _ESS ) as follows: If

, _{S M} is set to ES _HV , ESS is set to ES _D ; otherwise, ES _M is set to ES _D , and _ESS is set to ES _HV If ES _S is greater than ES _M , _Di is set to 0. Otherwise, if

ES _M *(ES _M +1)/2 + ES _S ; otherwise

ES _M *(ES _M +1)/2+ES _S +

2

The video decoder 300 may be configured to derive the class index C _i as

Based on C _i , the video transcoder 200 and the video decoder 300 may be configured to select filters from the filter bank F(f/s, i).

Various aspects of the second stage ALF 410 will now be described in more detail. In the second stage ALF 410, F' is the filter bank and C' is the corresponding classifier. The intermediate filter result can be further filtered with the current sample and its neighbors. C' can be used to decide which filter in F' to apply and how to transpose the coefficients. In one example, F' may be a signaled filter bank or a fixed (predefined) filter bank. In one example, C' may use Rs and/or R's to determine the filter index of filterbank F' by calculating activity and direction in the same manner as the first stage. Transpose can be applied when F' is applied.

,

）等）來選擇固定濾波器組F（f, i）和分類器C（f, i）。在一些實例中，對於每個F（f, i），可以用訊號發送語法元素以指示針對第i個固定濾波器組使用候選固定濾波器組中的哪個固定濾波器組。語法元素可以作為SPS、PPS、VPS、APS、圖片標頭（PH）、切片標頭（SH）、子圖片、CTU或塊的一部分用訊號進行發送。在一些實例中，所有這些語法元素可以具有相同的值並且只需要用訊號發送一個語法元素。在一些實例中，只有一個候選固定濾波器組可能可用於F（f, i），因此不需要用訊號發送語法元素。 The fixed filter bank F(f, i) and the corresponding classifier C(f, i) can be selected from one or more candidate fixed filter banks. In one example, given i, there may be some candidate fixed filter banks for fixed filter bank F(f, i) and classifier C(f, i). And can be based on coding information (eg, picture/CTU/CU/PU/TU size, quantization parameter (QP), number of non-zero quantization coefficients, motion vector, window size (

,

), etc.) to select a fixed filter bank F(f, i) and a classifier C(f, i). In some examples, for each F(f, i), a syntax element may be signaled to indicate which of the candidate fixed filter sets to use for the ith fixed filter set. Syntax elements may be signaled as part of SPS, PPS, VPS, APS, picture header (PH), slice header (SH), sub-picture, CTU or block. In some instances, all of these syntax elements may have the same value and only one syntax element needs to be signaled. In some examples, only one candidate fixed filter bank may be available for F(f, i), so no syntax element needs to be signaled.

In one example, _Ns and _Nf may be implicitly determined or explicitly signaled at a high level (eg, SPS, VPS, APS, PH, SH, CTU, or sub-block level).

Video decoder 300 may be configured to implement multiple fixed filters and one signaled filter. In another example, in order to reduce the management burden of bitstream signaling while maintaining self-adjustment, video transcoder 200 and video decoder 300 may be configured to use only one signaling with multiple fixed filters The filter filters the current sample, as shown in FIG. 11 .

11 illustrates an example framework for implementing multiple fixed filters with one signaled filter in accordance with the techniques of the present disclosure. In the example of FIG. 11, the video transcoder 200 and the video decoder 300 apply the first stage ALF 540 to the reconstructed samples R(x, y) of the reconstructed block. In order to apply the first stage ALF 540, the video transcoder 200 and the video decoder 300 apply a classifier (C) to decide a first class index for reconstructing samples, and based on the first class index from the first set of filters (F ) to select the filter (f). Video transcoder 200 and video decoder 300 apply filters from the first set of filters to the reconstructed samples to determine a first intermediate sample value (R'). As shown in FIG. 11, in the first stage 540, the video transcoder 200 and the video decoder 300 may use multiple classifiers and multiple filters to determine multiple intermediate sample values.

Video transcoder 200 and video decoder 300 apply second stage ALF 542 to reconstructed samples. Equation (28) below illustrates an example of how video transcoder 200 and video decoder 300 may apply second-stage ALF.

N _f represents the number of fixed (predefined) filter banks that can be applied to the samples. R s is the input to the ALF. F(f, i), where i = 0... N _f -1 denotes the ith fixed filter bank. C(f, i), where i = 0... N _f - 1 denotes the classifier with the ith fixed filter bank. Via C(f,i), for each sample, the filter index is calculated. Based on the filter index, a filter is selected from the fixed filter bank F(f, i) to filter the sample. Furthermore, C(f, i) decides how the geometric transformation is applied to the coefficients.

（19） The first stage will now be described. In the first stage, for reconstructed samples R(x, y), video transcoder 200 and video decoder 300 may be configured to apply multiple fixed filter banks by using the reconstructed samples themselves and adjacent samples. Given a fixed filter bank F(f, i), video transcoder 200 and video decoder 300 may apply a classifier C(f, i). Based on the derived class index, video transcoder 200 and video decoder 300 may apply filters from filter bank F(f, i). Furthermore, the geometric transformation is decided in C(f, i), and the geometric transformation can be applied to the coefficients of the selected filter. After the first stage, for each filter bank and corresponding classifier, the video transcoder 200 and the video decoder 300 may calculate the intermediate filtered signal R', eg, as follows:

(19)

時，

是從第i個濾波器組F（f, i）中選取的濾波器的第j個係數，

是係數的數量，並且

其中 k = 0或1是一個相鄰取樣和當前取樣的函數，例如，

可以被實施為如在VVC中的修剪函數

（20） （

是相鄰取樣相對於當前取樣的座標偏移量，

是從第i個濾波器組中選擇的濾波器的第j個修剪參數。 For the ith filter, where i = 0...

hour,

is the jth coefficient of the filter selected from the ith filter bank F(f, i),

is the number of coefficients, and

where k = 0 or 1 is a function of adjacent samples and the current sample, for example,

can be implemented as a trim function as in VVC

(20) (

is the coordinate offset of adjacent samples relative to the current sample,

is the jth trim parameter of the filter selected from the ith filter bank.

In some examples, the filter shapes of the fixed or signaled filters may be 5x5, 7x7, 9x9, 11x11 and 13x13 as shown in FIG. 9 .

In one example, video transcoder 200 and video decoder 300 may determine activity and orientation values in classifier C(f,i) based on 2-D Laplacian values. A classifier can be applied to each sample or block. When the classifier is applied to a block, all samples in the block have the same class index and the same transpose type. For example, _wi may represent the width of the block, _hi may represent the height of the block, and (x, y) may represent the coordinates of the upper-left corner sample of the block.

（21） For samples with coordinates (k, l), video transcoder 200 and video decoder 300 may be configured to derive four Laplacian (gradient) values: horizontal gradient H, vertical gradient V, 135 degree gradient D1 and 45 degree gradient D2, as follows:

(twenty one)

（22） 其中 a _i 和 b _i 分別是用於分類器C（f, i）的在水平和垂直方向上的訊窗大小。 The video transcoder 200 and the video decoder 300 can derive the activity value A _i by using the vertical and horizontal gradients as follows:

(22) where a _i and b _i are the window sizes in the horizontal and vertical directions for the classifier C(f, i), respectively.

The video transcoder 200 and the video decoder 300 may be configured to quantize A _i to a closed interval from 0 to M _A,i -1, and the quantized value is denoted as  _i .

Q[193] = {0,  1,  2,  3,  4,  4,  5,  5,  6,  6,  6,  6,  7,  7,  7,  7,  8,  8,  8,  8,  8,  8,  8,  8,  9,  9,  9,  9,  9,  9,  9,  9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15}； 位元深度是R（x, y）的位元深度，

可能取決於窗口大小（ a _i, b _i ），並且

可以等於以下值： a _i b _i mult _i 0 0 5628 1 1 1407 2 2 624 3 3 351 4 4 225 5 5 156 In one instance,

Q[193] = {0, 1, 2, 3, 4, 4, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 15}; bit depth is R(x , y) the bit depth,

may depend on the window size ( a _i , b _i ), and

Can be equal to the following values: a _i b _i mult _i 0 0 5628 1 1 1407 2 2 624 3 3 351 4 4 225 5 5 156

The video transcoder 200 and the video decoder 300 may be configured to determine the direction by using the horizontal gradient H , the vertical gradient V , the 135-degree gradient D1 and the 45-degree gradient D2 .

、垂直梯度

和兩個對角線梯度

和

的值，如下：

（23） The video transcoder 200 and the video decoder 300 may be configured to first calculate the horizontal gradient

, vertical gradient

and two diagonal gradients

and

value, as follows:

(twenty three)

（24）

（25） To assign the directivity D _i , the video transcoder 200 and the video decoder 300 may be configured to derive the maximum and minimum values of the horizontal and vertical gradients and the maximum and minimum values of the two diagonal gradients as follows:

(twenty four)

(25)

=5： 步驟1. 若

並且

，

被設置為 0（塊被歸類為「紋理」）。 步驟2. 若

，從步驟3繼續，否則從步驟4繼續。 步驟3. 若

，則

被設置為2（塊被歸類為「強的水平/垂直」），否則

被設置為1（塊被歸類為「弱的水平/垂直」）。 步驟4. 若

，則

被設置為4（塊被歸類為「強的對角線」），否則

被設置為3（塊被歸類為「弱的對角線」）。 Subsequently, the video transcoder 200 and the video decoder 300 may be configured to derive the direction D _i , and the number of directions as follows

=5: Step 1. If

and

,

is set to 0 (blocks are classified as "textures"). Step 2. If

, continue from step 3, otherwise continue from step 4. Step 3. If

,but

is set to 2 (blocks are classified as "strong horizontal/vertical"), otherwise

is set to 1 (blocks are classified as "weak horizontal/vertical"). Step 4. If

,but

is set to 4 (blocks are classified as "strong diagonal"), otherwise

is set to 3 (blocks are classified as "weak diagonal").

In some instances, more directions may be supported.

與閾值（Th）陣列進行比較，來計算水平/垂直方向（ES _i,HV）的邊緣強度。閾值陣列的大小可以是 S，並且閾值可以昇冪排列。方向的數量是

步驟1.初始化m = 0 並且 ES _i,HV= 0； 步驟2.若m等於 S，則停止；否則，轉到步驟3。 步驟3.若

＞Th[m], m = m+1並且ES _i,HV= ES _i,HV+ 1，進入步驟2；否則，停止。 The video transcoder 200 and the video decoder 300 may be configured to

The edge intensities in the horizontal/vertical direction (ES _i,HV ) are calculated by comparing with the threshold (Th) array. The size of the threshold array can be S , and the thresholds can be arranged in ascending powers. The number of directions is

Step 1. Initialize m = 0 and ES _i,HV = 0; Step 2. If m is equal to S, stop; otherwise, go to Step 3. Step 3. If

>Th[m], m = m+1 and ES _i,HV = ES _i,HV + 1, go to step 2; otherwise, stop.

＞Th[m]，m = m+1並且E _i,D= E _i,D+ 1，進入步驟2；否則，停止。 The video transcoder 200 and the video decoder 300 may be configured to calculate the diagonal direction (E _i,D ) as follows: Step 1. Initialize m = 0 and E _i,D = 0; Step 2. If m equals S, then stop; otherwise, go to step 3. Step 3. If

>Th[m], m = m+1 and E _{i, D} = E _{i, D} + 1, go to step 2; otherwise, stop.

An example of a threshold array is Th=[1.25, 1.5, 2, 3, 4.5, 8] and S=6.

, ES _M被設置為ES _HV並且ES _S被設置為ES _D；否則，ES _M被設置為ES _D，ES _S被設置為ES _HV若ES _S大於ES _M，則D _i被設置為0。否則，若

ES _M*（ES _M+1）/2 + ES _S；否則，

ES _M*（ES _M+1）/2 + ES _S+

2 類別索引

可以被推導為

基於

，從C（f, i）中挑選濾波器。 Video transcoder 200 and video decoder 300 may be configured to determine primary edge strength (ES _M ) and secondary edge strength ( _ESS ) as follows: If

, ES _M is set to ES _HV and ES _S is set to ES _D ; otherwise, ES _M is set to ES _D and ES _S is set to ES _HV . If ES _S is greater than ES _M , then _Di is set to 0. Otherwise, if

ES _M *(ES _M +1)/2 + ES _S ; otherwise,

ES _M *(ES _M +1)/2+ES _S +

2 Category Index

can be deduced as

based on

, pick a filter from C(f, i).

The video transcoder 200 and the video decoder 300 may select a fixed filter bank F(f,i) and a corresponding classifier C(f,i) from one or more candidate fixed filter banks. In one example, given i, there may be some candidate fixed filter banks for fixed filter bank F(f, i) and classifier C(f, i). And the fixed filter bank F can be selected based on decoding information, such as picture/CTU/CU/PU/TU size, QP, number of non-zero quantization coefficients, motion vector, window size ( a _i , b _i ), etc. (f, i) and the classifier C(f, i). In some examples, for each F(f, i), a syntax element may be signaled to indicate which of the candidate fixed filter sets for the i-th fixed filter set to use. Syntax elements may be signaled at the SPS, PPS, VPS, APS, PH, SH, sub-picture, CTU or block level. In some instances, all of these syntax elements may have the same value and only one syntax element needs to be signaled. In some examples, a candidate fixed filter bank may have only one fixed filter bank, so no syntax element needs to be signaled.

可以用訊號發送語法元素以指示是將第 I ₀個候選固定濾波器組還是第 I ₁個候選固定濾波器組用於F（f, i）。在一些實例中，所有固定濾波器組可以共享相同的語法元素。 In one example, given the window size ( a _i , b _i ), the number of all available fixed filters is T. For example, QP may represent the quantization parameter of the current sample, where QP_MIN, QP_MAX, and QP_OFFSET are predefined values. The indices of the two candidate fixed filter banks can be determined as

A syntax element may be signaled to indicate whether to use the 10th candidate fixed filter set or the 11th candidate fixed filter set for _F ( _f , i ). In some instances, all fixed filter banks may share the same syntax elements.

Various aspects of the second stage will now be described. In the second stage, F' is the signaled filter or predefined filter bank, and C' is the corresponding classifier. The intermediate filtering result may be further filtered with the current sample and/or its neighbors. C' can be used to decide which filter in F' to apply and how to transpose the coefficients. C' may use R s and/or R' s to determine filter indices for filter bank F' by computing activity and direction as a first stage. Transpose can be applied when F' is applied.

（26） In one example, for a sample with coordinates (k, l), four Laplacian (gradient) values: horizontal gradient H , vertical gradient V , 135 degree gradient D1 and 45 degree gradient D2 can be derived as

(26)

（27） 其中 a和 b分別是針對分類器C’在水平和垂直方向上的訊窗大小，並且 w和 h是其中所有取樣具有相同分類索引和相同轉置索引的塊的寬度和高度。 The video transcoder 200 and the video decoder 300 may be configured to derive the activity value A by using the vertical and horizontal gradients as follows:

(27) where a and b are the window sizes in the horizontal and vertical directions for classifier C', respectively, and w and h are the width and height of the block where all samples have the same classification index and the same transpose index.

的 A進行量化，並且經量化的值被表示為

。例如，

位元深度是R（x, y）的位元深度， mult可以取決於訊窗大小 a*b，mult可以等於以下值： a b mult 0 0 5628 1 1 1407 2 2 624 3 3 351 4 4 225 5 5 156 The video transcoder 200 and the video decoder 300 can be configured to

A is quantized, and the quantized value is expressed as

. E.g,

The bit depth is the bit depth of R(x, y), mult can depend on the window size a*b , mult can be equal to the following values: a b mult 0 0 5628 1 1 1407 2 2 624 3 3 351 4 4 225 5 5 156

Orientation and how to perform the transpose may be calculated via the manner described in US Patent Publication 2017/0238020A1, US Patent Publication 2017/0237981A1, or US Patent Publication 2017/0237982A1, all of which are incorporated herein by reference.

（28） 在（28）中，濾波被分為兩部分： 濾波部分1：

：使用相鄰取樣進行濾波，可以應用幾何變換。 N ₀是被應用的濾波器形狀的係數數量。濾波器可以是5x5、7x7、9x9、11x11或13x13菱形濾波器，如圖9中所示。 濾波部分2：

：經由使用中間濾波取樣進行濾波，可以應用幾何變換。 N ₀和 N ₁表示潛在唯一係數的數量。 函數

，其中j = 0或1，可以用修剪函式定義為   

（29） 函數

可以用修剪函式定義為

（30） b _i 是係數c _i相對應的修剪參數。 Various aspects of filtering will now be described. After obtaining the sampled class index from C', video transcoder 200 and video decoder 300 may be configured to select filters from filter bank F' based on the class index. Video transcoder 200 and video decoder 300 may be configured to apply filters as:

(28) In (28), the filtering is divided into two parts: Filtering part 1:

: Filtering using adjacent samples, geometric transformations can be applied. N ₀ is the number of coefficients of the filter shape being applied. The filter can be a 5x5, 7x7, 9x9, 11x11 or 13x13 diamond filter as shown in Figure 9. Filtering part 2:

: A geometric transformation can be applied via filtering using intermediate filtered samples. N ₀ and N ₁ represent the number of potential unique coefficients. function

, where j = 0 or 1, can be defined with the trim function as

(29) function

can be defined with the trim function as

(30) b _i is the trimming parameter corresponding to the coefficient c _i .

12A-12E illustrate examples of two-part filters. FIG. 12A illustrates an example where filter section 1 (filter 560 ) is 5x5, N ₁ =7, and for filter section 2 (filter 562 ), N ₁ =2. Figure 12B illustrates an example where filter section 1 (filter 564) is 7x7, N ₀ =13, and for filter section 2 (filter 566 ), N ₁ =2. FIG. 12C illustrates an example where filter section 1 (filter 568 ) is 9x9, N ₁ =21, and for filter section 2 (filter 570 ), N ₁ =2. FIG. 12D illustrates an example where filter part 1 (filter 572 ) is 11×11, N ₁ =31, and for filter part 2 (filter 574 ), N ₁ =2. FIG. 12E illustrates an example where filter section 1 (filter 576 ) is 13x13, N ₁ =43, and for filter section 2 (filter 578 ), N ₁ =2.

In some examples, the values of N0 and/or N1 may be explicitly signaled or implicitly determined _at the SPS, VPS, PPS, APS, PH, SH, sub _- picture header, or block level.

以及修剪參數

沒有用訊號發送/使用，並且被推斷為0。 In some examples, a flag may be explicitly signaled or implicitly determined at the SPS, VPS, PPS, APS, PH, SH, sub-picture header, block level, or filter level such that when the flag equals a value , all coefficients

and pruning parameters

Not signaled/used and inferred to be 0.

和修剪參數

沒有被用訊號發送/使用，並且被推斷為 0。 In some examples, a flag may be explicitly signaled or implicitly determined at the SPS, VPS, PPS, APS, PH, SH, sub-picture header, block level, or filter level such that when the flag equals a value , all coefficients

and trim parameters

Not signaled/used and inferred to be 0.

Within a set of filters, the value of N ₀ can vary from filter to filter. Within _a filter bank, the value of N1 can vary from filter to filter.

The subject matter recognizes that ALF performance can be improved over the techniques described in JVET-U0100 and US Provisional Patent Application No. 63/130,275. When only filtered samples from the first stage are preferred and no filtering is required in the second stage, the filter coefficients _ci (which are all zeros) of F' are still signaled, where i=0... N ₀ - 2. However, it is not necessary to signal all these zero coefficients. Therefore, alternative signaling procedures can be applied to improve decoding efficiency.

In some examples, video transcoder 200 and video decoder 300 may be configured to encode data in a bitstream representing one or more filters F(f,i), rather than using only fixed filtering device. Additionally or alternatively, the video transcoder 200 and the video decoder 300 may be configured to decide whether to perform the second stage filtering as discussed above. When the second stage is not performed, the video transcoder 200 and the video decoder 300 may avoid coding (encoding and/or decoding) the coefficients for one or more filters applied during the second stage, which may reduce The signaling management burden in the bitstream and the processing performed by both video transcoder 200 and video decoder 300 .

13 is a conceptual diagram illustrating an example ALF framework in accordance with techniques disclosed herein. Video decoder 300 may be configured to perform the techniques described with reference to FIG. 13 . As shown in FIG. 13 , the example ALF framework includes both a first stage 580 and a second stage 582 . According to the techniques of the present disclosure, the video transcoder 200 and the video decoder 300 can use the filter usage information to decide whether to perform the second stage. When the second stage is not performed, the video transcoder 200 and the video decoder 300 can bypass the filtering (C', F') in the second stage, as shown in FIG. 13 . Therefore, video transcoder 200 and video decoder 300 can use only intermediate filtered samples from the first stage (R'(x,y,0)...R'(x,y,Nf-1)) to calculate filtered of sampling.

（21） 其中

是

的權重。 Additionally, video decoder 300 may calculate a weighted sum of intermediate filtered samples from various filters, for example, according to the following formula:

(21) of which

Yes

the weight of.

In some examples, filter usage information may be signaled at the sequence, picture, slice, subpicture, CTU, CTB, or subblock level. For example, video transcoder 200 and video decoder 300 may encode filter usage information in one or more of the following: sequence parameter set (SPS), picture parameter set (PPS), slice header, Sub-picture header, coding tree unit (CTU) header, coding tree block (CTB) header, or sub-block header.

In another example, the filter usage information may not be signaled. Rather, video decoder 300 may self-adjustably derive filter usage information based on coding information, such as quantization parameters (QPs), block sizes, inter/intra-frame modes, or other such data.

（其中 i= 0… N _f - 1）可以是固定的，或者視訊轉碼器200和視訊解碼器300可以在序列、圖片、切片、子-圖片、CTU、CTB、子塊級對位元串流中的權重 w _i 的值進行編碼。可以使用權重 w _i 的若干個組合，可以使用濾波器使用資訊來決定權重 w _i 的哪個組合被用於當前取樣。 The value of the given filter usage information, the weight

(where i = 0... N _f - 1) may be fixed, or video transcoder 200 and video decoder 300 may align bit strings at sequence, picture, slice, sub-picture, CTU, CTB, sub-block level The value of the weight w _i in the stream is encoded. Several combinations of weights _wi can be used, and filter usage information can be used to decide which combination of weights _wi is used for the current sample.

In one example, the first flag can be signaled. When the first flag is equal to 1, then the video transcoder 200 and the video decoder 300 can bypass the value (C', F'). Subsequently, the second flag can be signaled. Video transcoder 200 and video decoder 300 may associate the value of the second flag with a combination of weights w _i (where i = 0 . . . N _f − 1 ). For example, if the second flag is equal to i , i may be 0... N _f - 1, then R'(x, y, i) may be used as filtered samples.

14 is a conceptual diagram illustrating another example ALF framework in accordance with techniques within the context of this disclosure. Video transcoder 200 and video decoder 300 may be configured according to the techniques described with reference to FIG. 14 . In particular, in Figure 14, scaling is applied to the filtered results to obtain filtered samples.

14 illustrates an example framework for filtering reconstructed samples of video data using multiple fixed filter banks and multiple signaled filter banks. Video decoder 300 may be configured to implement multiple fixed filters and multiple signaled filters in accordance with the techniques of FIG. 14 . In the example of Figure 14, the video decoder 300 applies the first stage ALF 584 to the reconstructed samples R(x,y) to determine the intermediate filtered signal R'. Subsequently, the video decoder 300 applies the second ALF stage 586 to R&apos; to determine the value of the filtered samples 588.

FIG. 15 is a block diagram illustrating an exemplary video transcoder 200 that may perform the techniques disclosed herein. Figure 15 is provided for purposes of explanation and should not be considered limiting of the techniques broadly illustrated and described in the context of this case. For explanatory purposes, the video transcoder 200 is described in accordance with the techniques of VVC (ITU-T H.266 under development) and HEVC (ITU-T H.265). However, the techniques of the present disclosure may be performed by video encoding devices configured to other video coding standards.

In the example of FIG. 15 , the video transcoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, Reconstruction unit 214 , filter unit 216 , DPB 218 , and entropy encoding unit 220 . Video data memory 230, mode selection unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, DPB 218, and entropy Any or all of encoding units 220 may be implemented in one or more processors or in processing circuits. For example, the elements of video transcoder 200 may be implemented as one or more circuits or logic components, as part of a hardware circuit, or as part of a processor, ASIC or FPGA. Furthermore, video transcoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

Video data memory 230 may store video data to be encoded by components of video transcoder 200 . Video transcoder 200 may receive video data stored in video data memory 230 from, for example, video source 104 (FIG. 1). DPB 218 may act as a reference picture memory that stores reference video data for use in prediction of subsequent video data by video transcoder 200 . Video data memory 230 and DPB 218 may be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM ( RRAM), or other types of memory devices. Video data memory 230 and DPB 218 may be provided by the same memory device or by separate memory devices. In various examples, video data memory 230 may be on-chip with other components of video transcoder 200 (as shown), or off-chip relative to those components.

In the context of this case, references to video data memory 230 should not be construed as limited to memory internal to video transcoder 200 (unless so specifically described), or to memory external to video transcoder 200 (unless so specifically described). Rather, references to video data memory 230 should be understood as reference memory that stores video data received by video transcoder 200 for encoding (eg, video data for the current block to be encoded). The memory 106 of FIG. 1 may also provide temporary storage of outputs from the various units of the video transcoder 200 .

The various elements of FIG. 15 are illustrated to illustrate understanding of the operations performed by the video transcoder 200 . These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Fixed-function circuits represent circuits that provide specific functions and are preset with respect to operations that can be performed. Programmable circuits represent circuits that can be programmed to perform various tasks, and that provide flexible functionality in the operations that can be performed. For example, the programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (eg, to receive parameters or output parameters), but the types of operations performed by fixed-function circuits are generally immutable. In some instances, one or more of the units may be distinct circuit blocks (fixed function or programmable), and in some instances, one or more of the units may be integrated circuit.

The video transcoder 200 may include an arithmetic logic unit (ALU), an elementary functional unit (EFU), a digital circuit, an analog circuit, and/or a programmable core formed of programmable circuits. In examples in which the operations of video transcoder 200 are performed using software executed by programmable circuitry, memory 106 (FIG. 1) may store instructions (eg, object code) for the software received and executed by video transcoder 200 ), or another memory (not shown) in the video transcoder 200 can store such instructions.

The video data memory 230 is configured to store the received video data. The video transcoder 200 can retrieve the pictures of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202 . The video data in the video data memory 230 may be the original video data to be encoded.

Mode selection unit 202 includes motion estimation unit 222 , motion compensation unit 224 and intra-frame prediction unit 226 . Mode selection unit 202 may include additional functional units that perform video prediction according to other prediction modes. As examples, mode selection unit 202 may include a palette unit, an intra-block copy unit (which may be part of motion estimation unit 222 and/or motion compensation unit 224), an affine unit, a linear model (LM) unit, and the like.

Mode selection unit 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and resulting rate-distortion values for such combinations. Coding parameters may include partitioning of the CTU into CUs, prediction mode for the CU, transform type for the residual data of the CU, quantization parameters for the residual data of the CU, and the like. The mode selection unit 202 may finally select a combination of encoding parameters that has a better rate-distortion value than the other tested combinations.

The video transcoder 200 may partition the pictures retrieved from the video data memory 230 into a series of CTUs, and encapsulate one or more CTUs in slices. The mode selection unit 202 may divide the CTUs of the picture according to a tree structure such as the above-described QTBT structure or quad-tree structure of HEVC. As described above, the video transcoder 200 may form one or more CUs by dividing the CTUs according to the tree structure. Such CUs may also often be referred to as "video blocks" or "blocks."

Typically, mode selection unit 202 also controls its components (eg, motion estimation unit 222, motion compensation unit 224, and intra-frame prediction unit 226) to generate a overlapping part) of the prediction block. To perform inter-frame prediction on the current block, motion estimation unit 222 may perform a motion search to identify a motion in one or more reference pictures (eg, one or more previously coded pictures stored in DPB 218). or multiple closely matched reference blocks. Specifically, the motion estimation unit 222 may, for example, based on sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute differences (MAD), mean square deviations (MSD), etc., to value of similar degree. Motion estimation unit 222 may typically perform these calculations using sample-by-sample differences between the current block and the reference block under consideration. Motion estimation unit 222 may identify the reference block with the lowest value resulting from these calculations, which indicates the reference block that most closely matches the current block.

Motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion estimation unit 222 may then provide the motion vector to motion compensation unit 224 . For example, for uni-directional inter-frame prediction, motion estimation unit 222 may provide a single motion vector, while for bi-directional inter-frame prediction, motion estimation unit 222 may provide two motion vectors. Subsequently, motion compensation unit 224 may use the motion vector to generate a prediction block. For example, motion compensation unit 224 may use motion vectors to retrieve information for reference blocks. As another example, if the motion vector has fractional sampling precision, motion compensation unit 224 may interpolate the values used for the prediction block according to one or more interpolation filters. Furthermore, for bi-directional inter-frame prediction, motion compensation unit 224 may retrieve data for the two reference blocks identified by corresponding motion vectors and combine the retrieved data, eg, via sample-by-sample averaging or weighted averaging.

As another example, for intra-frame prediction or intra-frame prediction coding, intra-frame prediction unit 226 may generate a prediction block from samples adjacent to the current block. For example, for directional mode, intra-frame prediction unit 226 may typically mathematically combine the values of adjacent samples and fill in these computed values in the defined direction across the current block to produce a predicted block. As another example, for DC mode, intra-frame prediction unit 226 may calculate an average of adjacent samples of the current block and generate a prediction block to include this resulting average for each sample of the prediction block.

The mode selection unit 202 provides the prediction block to the residual generation unit 204 . Residual generation unit 204 receives the original unencoded version of the current block from video data memory 230 and receives the prediction block from mode selection unit 202 . The residual generation unit 204 calculates a sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample differences define the residual block for the current block. In some examples, residual generation unit 204 may use differences between sample values in the residual block to generate the residual block using residual differential pulse decoding modulation (RDPCM). In some examples, residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

In examples in which mode selection unit 202 partitions the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. Video transcoder 200 and video decoder 300 may support PUs with various sizes. As noted above, the size of the CU may represent the size of the CU's luma coding block, while the size of the PU may represent the size of the PU's luma prediction unit. Assuming the size of a particular CU is 2Nx2N, video transcoder 200 may support PU sizes of 2Nx2N or NxN for intra-frame prediction, and 2Nx2N, 2NxN, Nx2N, NxN or similar symmetric for inter-frame prediction PU size. Video transcoder 200 and video decoder 300 may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter-frame prediction.

In examples where the mode selection unit does not further partition the CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As previously described, the size of the CU may represent the size of the luma coding block of the CU. The video transcoder 200 and the video decoder 300 may support a CU size of 2Nx2N, 2NxN or Nx2N.

For other video coding techniques (such as intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, to name a few examples), mode selection unit 202 selects via corresponding units associated with the coding technique A prediction block is generated for the current block being encoded. In some examples (such as palette mode coding), mode selection unit 202 may not generate prediction blocks, but instead generate syntax elements that indicate how the blocks are to be reconstructed based on the selected palette. In such modes, mode selection unit 202 may provide these syntax elements to entropy encoding unit 220 for encoding.

As previously described, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. Subsequently, the residual generating unit 204 generates a residual block for the current block. To generate a residual block, the residual generation unit 204 calculates a sample-by-sample difference between the prediction block and the current block.

Transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transforms to the residual block to form a block of transform coefficients. For example, transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, transform processing unit 206 may perform various transforms, eg, primary transforms and secondary transforms (such as rotational transforms), on the residual block. In some examples, transform processing unit 206 does not apply a transform to the residual block.

Quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to the QP value associated with the current block. Video transcoder 200 (eg, via mode selection unit 202 ) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may cause a loss of information, and thus, the quantized transform coefficients may have lower precision than the original transform coefficients produced by transform processing unit 206 .

Inverse quantization unit 210 and inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. Reconstruction unit 214 may generate a reconstructed block corresponding to the current block (albeit potentially with some degree of distortion) based on the reconstructed residual block and the prediction block produced by mode selection unit 202 . For example, reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the prediction block generated by mode selection unit 202 to generate a reconstructed block.

Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform deblocking operations to reduce blocking artifacts along edges of the CU. In some examples, the operations of filtering unit 216 may be skipped.

Video transcoder 200 stores the reconstructed blocks in DPB 218 . For example, in instances in which the operations of filter unit 216 are not performed, reconstruction unit 214 may store the reconstructed block into DPB 218 . In examples in which the operations of filter unit 216 are performed, filter unit 216 may store the filtered reconstructed block into DPB 218 . Motion estimation unit 222 and motion compensation unit 224 may retrieve reference pictures formed from reconstructed (and potentially filtered) blocks from DPB 218 for inter-frame prediction of blocks of subsequently encoded pictures. Additionally, intra-prediction unit 226 may use the reconstructed blocks of the current picture in DPB 218 to intra-predict other blocks in the current picture.

In general, entropy encoding unit 220 may entropy encode syntax elements received from other functional components of video transcoder 200 . For example, entropy encoding unit 220 may entropy encode the quantized block of transform coefficients from quantization unit 208. As another example, entropy encoding unit 220 may entropy encode prediction syntax elements from mode selection unit 202 (eg, motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction). Entropy encoding unit 220 may perform one or more entropy encoding operations on syntax elements, which are another example of video data, to generate entropy encoded data. For example, the entropy encoding unit 220 may perform context self-adjusting variable length coding (CAVLC) operations, CABAC operations, variable-variable (V2V) length coding operations, syntax-based context self-adjusting binary arithmetic coding (SBAC) operation, Probability Interval Partitioning Entropy (PIPE) decoding operation, Exponential Golomb encoding operation, or another type of entropy encoding operation on the data. In some examples, entropy encoding unit 220 may operate in a bypass mode in which syntax elements are not entropy encoded.

Video transcoder 200 may output a bitstream that includes entropy-encoded syntax elements needed to reconstruct blocks of a slice or picture. Specifically, the entropy encoding unit 220 may output a bitstream.

The above operations are described with respect to blocks. Such descriptions should be understood as operations for luma coding blocks and/or chroma coding blocks. As previously described, in some examples, the luma coding blocks and the chroma coding blocks are the luma and chroma components of a CU. In some examples, the luma coding blocks and the chroma coding blocks are the luma and chroma components of the PU.

In some examples, operations performed with respect to luma coding blocks need not be repeated for chroma coding blocks. As one example, operations to identify motion vectors (MVs) and reference pictures for luma coded blocks need not be multiplexed to identify MVs and reference pictures for chroma blocks. Specifically, the MVs for luma coded blocks may be scaled to decide the MVs for chroma blocks, and the reference pictures may be the same. As another example, the intra-frame prediction procedure may be the same for luma coded blocks and chroma coded blocks.

FIG. 16 is a block diagram illustrating an exemplary video decoder 300 that may perform the techniques disclosed herein. Figure 16 is provided for purposes of explanation and is not intended to limit the techniques generally illustrated and described in the context of this case. For explanatory purposes, the video decoder 300 is described in terms of VVC (ITU-T H.266 under development) and HEVC (ITU-T H.265) techniques. However, the techniques of this disclosure may be performed by video coding devices configured for other video coding standards.

In the example of FIG. 16, video decoder 300 includes coded picture buffer (CPB) memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, Filter unit 312 and DPB 314. Any or all of CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 134 may be one or more implemented in a processor or in a processing circuit. For example, the elements of video decoder 300 may be implemented as one or more circuits or logic components, as part of a hardware circuit, or as part of a processor, ASIC or FPGA. Furthermore, video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

Prediction processing unit 304 includes motion compensation unit 316 and intra-frame prediction unit 318 . Prediction processing unit 304 may include additional units that perform prediction according to other prediction modes. As examples, prediction processing unit 304 may include a palette unit, an intra-block copy unit (which may form part of motion compensation unit 316), an affine unit, a linear model (LM) unit, and the like. In other examples, video decoder 300 may include more, fewer, or different functional components.

CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of video decoder 300 . For example, video data stored in CPB memory 320 may be obtained from computer readable medium 110 (FIG. 1). CPB memory 320 may include a CPB that stores encoded video data (eg, syntax elements) from the encoded video bitstream. Additionally, CPB memory 320 may store video data other than syntax elements of the decoded picture, such as temporary data representing outputs from various units of video decoder 300 . DPB 314 typically stores decoded pictures, which video decoder 300 may output and/or use as reference video data when decoding subsequent data or pictures of the encoded video bitstream. CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices, such as DRAM (including SDRAM), MRAM, RRAM, or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or by separate memory devices. In various examples, CPB memory 320 may be on-chip with other components of video decoder 300, or off-chip relative to those components.

Additionally or alternatively, in some examples, video decoder 300 may retrieve decoded video data from memory 120 (FIG. 1). That is, memory 120 may utilize CPB memory 320 to store data as discussed above. Likewise, when some or all of the functions of the video decoder 300 are implemented in software to be executed by the processing circuitry of the video decoder 300 , the memory 120 may store instructions to be executed by the video decoder 300 .

The various elements shown in FIG. 16 are illustrated to illustrate understanding of the operations performed by the video decoder 300 . These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Similar to FIG. 15, fixed function circuits represent circuits that provide specific functions and are preset with respect to operations that can be performed. Programmable circuits represent circuits that can be programmed to perform various tasks and provide flexible functionality in operations that can be performed. For example, the programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits may execute software instructions (eg, to receive parameters or output parameters), but the type of operations performed by fixed-function circuits is generally immutable. In some instances, one or more of the units may be distinct circuit blocks (fixed function or programmable), and in some instances, one or more of the units may be integrated circuits.

Video decoder 300 may include ALUs, EFUs, digital circuits, analog circuits, and/or programmable cores formed from programmable circuits. In instances where the operations of video decoder 300 are performed by software executing on programmable circuitry, on-chip or off-chip memory may store instructions (eg, object code) for the software that video decoder 300 receives and executes.

Entropy decoding unit 302 may receive encoded video data from the CPB and entropy decode the video data to reproduce syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, and filter unit 312 may generate decoded video data based on syntax elements extracted from the bitstream.

Typically, the video decoder 300 reconstructs the picture block by block. Video decoder 300 may perform reconstruction operations on each block individually (where the block currently being reconstructed (ie, decoded) may be referred to as the "current block").

Entropy decoding unit 302 may entropy decode syntax elements that define the quantized transform coefficients of the quantized transform coefficient block and transform information such as quantization parameters (QPs) and/or transform mode indications. Inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine the degree of quantization, and likewise, the degree of inverse quantization for inverse quantization unit 306 to apply. Inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. Inverse quantization unit 306 may thus form a block of transform coefficients comprising transform coefficients.

After inverse quantization unit 306 forms the block of transform coefficients, inverse transform processing unit 308 may apply one or more inverse transforms to the block of transform coefficients to generate a residual block associated with the current block. For example, inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the transform coefficient block.

In addition, prediction processing unit 304 generates prediction blocks from prediction information syntax elements entropy decoded by entropy decoding unit 302 . For example, motion compensation unit 316 may generate a prediction block if the prediction information syntax element indicates that the current block is inter-frame predicted. In this case, the prediction information syntax element may indicate the reference picture in DPB 314 from which the reference block is to be retrieved, and identify the position of the reference block in the reference picture relative to the position of the current block in the current picture motion vector. Motion compensation unit 316 may generally perform an inter-frame prediction procedure in a manner substantially similar to that described with respect to motion compensation unit 224 (FIG. 15).

As another example, if the prediction information syntax element indicates that the current block is intra-predicted, intra-frame prediction unit 318 may generate the prediction block according to the intra-frame prediction mode indicated by the prediction information syntax element. Again, intra-frame prediction unit 318 may generally perform the intra-frame prediction procedure in a manner substantially similar to that described with respect to intra-frame prediction unit 226 (FIG. 5). In-frame prediction unit 318 may retrieve data from DPB 314 for adjacent samples of the current block.

The reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

Filter unit 312 may perform one or more filter operations on the reconstructed block. For example, filter unit 312 may perform a deblocking operation to reduce blocking artifacts along edges of reconstructed blocks. The operations of filter unit 312 are not necessarily performed in all instances.

Video decoder 300 may store the reconstructed blocks in DPB 314 . For example, reconstruction unit 310 may store the reconstructed block to DPB 314 in instances where the operations of filter unit 312 are not performed. In instances where the operations of filter unit 312 are performed, filter unit 312 may store the filtered reconstructed block to DPB 314 . As discussed above, DPB 314 may provide reference information, such as the current picture for intra-frame prediction and samples of previously decoded pictures for subsequent motion compensation, to prediction processing unit 304 . Furthermore, video decoder 300 may output decoded pictures (eg, decoded video) from DPB 314 for subsequent presentation on a display device, such as display device 118 of FIG. 1 .

FIG. 17 illustrates an exemplary implementation of the filter unit 312 in FIG. 16 . The filter unit 216 in Figure 15 may be implemented via the same approach. Filter units 216 and 312 may implement the techniques described herein, possibly in conjunction with other components of video transcoder 200 or video decoder 300. In the example of FIG. 16 , filter unit 312 includes deblocking filter 342 , SAO filter 344 , and ALF unit 346 . The SAO filter 344 may, for example, be configured to decide offset values for sampling of the block in the manner described in the context of this application. ALF unit 346 may also filter blocks of video data in the manner described in the context of this application. For example, the ALF unit 346 may be configured to apply a first stage ALF to reconstructed samples of the reconstructed block to determine a first sample modification value, apply a second stage ALF to the reconstructed samples to determine a second sample modification value, and, based on The reconstructed samples, the first sample modification value, and the second sample modification value determine the filtered reconstructed samples.

Filter unit 312 may include fewer filters and/or may include additional filters. Furthermore, the particular filters shown in Figure 17 may be implemented in a different order. Other in-loop filters (in the decoding loop, or after the decoding loop) can also be used to smooth primitive transitions or otherwise improve video quality. The filtered reconstructed video blocks output by filter unit 312 may be stored in DPB 314, which stores reference pictures for subsequent motion compensation. DPB 314 may be part of or separate from additional memory that stores decoded video for later presentation on a display device (eg, display device 118 of FIG. 1 ).

18 is a flow diagram illustrating an example procedure for encoding a current block in accordance with the techniques of this disclosure. The current block may include the current CU. Although described with respect to the video transcoder 200 ( FIGS. 1 and 14 ), it should be understood that other devices may be configured to perform a procedure similar to that of FIG. 18 .

In this example, video transcoder 200 initially predicts the current block (650). For example, video transcoder 200 may form a prediction block for the current block. Subsequently, video transcoder 200 may calculate a residual block for the current block (652). To calculate the residual block, video transcoder 200 may calculate the difference between the original unencoded block and the predicted block for the current block. Then, video transcoder 200 may transform the residual block and quantize the transform coefficients of the residual block (654). Next, video transcoder 200 may scan the residual block for the quantized transform coefficients (656). During or after the scan, video transcoder 200 may entropy encode the transform coefficients (658). For example, video transcoder 200 may use CAVLC or CABAC to encode transform coefficients. Video transcoder 200 may then output entropy encoded data for the block (660).

19 is a flowchart illustrating an example method for decoding a current block of video material in accordance with the techniques of this disclosure. The current block may include the current CU. Although described with respect to video decoder 300 (FIGS. 1 and 15), it should be understood that other devices may be configured to perform a procedure similar to that of FIG. 19 .

Video decoder 300 may receive entropy-encoded data for the current block (eg, entropy-encoded prediction information and entropy-encoded data for transform coefficients of residual blocks corresponding to the current block) (670) . Video decoder 300 may entropy decode the entropy encoded data to determine prediction information for the current block and reproduce transform coefficients for the residual block (672). Video decoder 300 may predict the current block, eg, using an intra-frame or inter-frame prediction mode as indicated by the prediction information for the current block (674), in order to calculate a prediction block for the current block. Video decoder 300 may then inverse scan (676) the rendered transform coefficients to create a block of quantized transform coefficients. Video decoder 300 may then inverse quantize the transform coefficients and apply an inverse transform to the transform coefficients to generate a residual block (678). Video decoder 300 may finally decode the current block by combining the prediction block and the residual block (680).

20 is a flow diagram illustrating an example procedure for decoding a current block of video data in accordance with the techniques disclosed herein. Although described with reference to video decoder 300 (FIGS. 1 and 15), it should be understood that other devices may be configured to perform a procedure similar to that of FIG. For example, the video decoding loop of video transcoder 200 including filter unit 216 may perform the technique of FIG. 20 .

The video decoder 300 applies the first-stage ALF to reconstructed samples of the reconstructed block, wherein applying the first-stage ALF includes (710). In this context, reconstructed samples may be reconstructed samples before deblock filtering or after deblock filtering. To apply the first stage ALF, the video decoder 300 determines a first class index for reconstructed samples (712), selects a filter from a first set of filters based on the first class index (714), and applies to the reconstructed samples Filters from the first set of filters to determine first intermediate sample values (716). As previously described, the video decoder 300 may select the first set of filters from the complex set of fixed filters based on the quantization parameters used to reconstruct the block. For example, the first set of filters may be fixed filters. That is, the first set of filters may be a set of filters stored in the video transcoder rather than signaled in the video data. The first set of filters may, for example, comprise at least one 9x9 diamond filter or filters of any size or shape described above. As shown in Figures 9 and 11, the video decoder can determine multiple intermediate sample values.

，其中S = 6意味著有56個方向。 To determine the first class index, the video decoder 300 may determine the activity value for reconstructing the sample, determine the direction for the reconstruction sample, and determine a classifier based on the activity value and direction. As before, the number of directions can be

, where S = 6 means there are 56 directions.

Video decoder 300 applies the second stage ALF to reconstructed samples (720). To apply the second stage ALF, the video decoder 300 determines a second class index for reconstructing samples (722), selects a second filter from a second set of filters based on the second class index (724), A processor is applied to the reconstructed samples to determine a first sample modification value (726), and, based on the first intermediate sample value, to determine a second sample modification value. The second set of filters may eg be signaled filters, which means that the filters in the second set are signaled at least partly in the video data.

Video decoder 300 determines filtered reconstructed samples based on the reconstructed samples, the first sample modification value, and the second sample modification value (730). To determine the filtered reconstructed samples based on the reconstructed samples, the first sample modification value, and the second sample modification value, video decoder 300 may, for example, add the first sample modification value and the second sample modification value to the reconstructed samples. Equation (28) above represents one example of how video decoder 300 may decide the values for the filtered reconstructed samples.

To determine the second sample modification value based on the first intermediate sample value, video decoder 300 may trim the second sample modification value to determine the trimmed sample modification value, and add the first sample modification value and the trimmed sample modification value to reconstruction sampling. Video decoder 300 may receive trimmed sample modification values in video data. In order to determine the second sample modification value based on the first intermediate sample value, the video decoder 300 may also determine the second sample modification value based on the first intermediate sample value by determining the difference between the reconstructed sample and the first intermediate sample value . Equations (28) and (30) above represent examples of how the video decoder 300 may determine the second sample modification value.

Video decoder 300 may output decoded pictures that include filtered reconstructed sampled video data. In some coding scenarios, video decoder 300 may perform one or more additional filtering operations on the filtered reconstructed samples prior to output. The video decoder 300 may store the picture in a decoded picture buffer for decoding subsequent pictures, store the decoded picture in a storage medium for subsequent display, or by outputting the decoded picture in real time to a display device or near real time display , so as to output the decoded picture. 20 is performed by a video transcoder, but the video transcoder may output decoded pictures, eg, by storing the pictures in a decoded picture buffer for encoding subsequent pictures.

The following numbered clauses describe one or more aspects of the devices and techniques described in the content of this case.

Clause 1A. A method of decoding video data, the method comprising: determining a reconstructed block of video data; and applying a filter to samples of the reconstructed block of video data to generate a filtered reconstructed block.

Clause 2A. The method of Clause 1A, wherein applying the filter to sampling comprises: determining one or both of an activity value for sampling and a direction for sampling; and, based on the determined activity value and/or or Direction, to select a filter from a set of filters.

Clause 3A. The method of Clause 2A, wherein the set of filters comprises a set of fixed filters.

Clause 4A. The method of clause 2A, wherein the set of filters comprises a set of signaled filters.

Clause 5A. The method of any of Clauses 1A-4A, wherein applying a filter to the samples comprises: applying a first filter to the samples to determine intermediate filtered samples; and, applying a second filter to the intermediate filtering Samples to determine the final filter of the samples.

Clause 6A. The method of any of clauses 1A-5A, wherein the decoding method is performed as part of a video encoding procedure.

Clause 7A. An apparatus for decoding video material, the apparatus comprising one or more means for performing the method of any of clauses 1A-6A.

Clause 8A. The apparatus of clause 7A, wherein the one or more units comprise one or more processors implemented in a circuit.

Clause 9A. The apparatus of any of clauses 7A and 8A, further comprising: memory for storing the video data.

Clause 10A. The apparatus of any of clauses 7A-9A, further comprising: a display configured to display the decoded video data.

Clause 11A. The apparatus of any of Clauses 7A-10A, wherein the apparatus comprises one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 12A. The apparatus of any of clauses 7A-11A, wherein the apparatus comprises a video decoder.

Clause 13A. The apparatus of any of clauses 7A-12A, wherein the apparatus comprises a video transcoder.

Clause 14A. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of Clauses 1A-6A.

Clause 1B. A method of decoding video data, the method comprising: decoding at least a portion of a picture of the video data; applying one or more ALFs to the picture during a first stage of a self-adjusting loop filter (ALF) at least part of the image to generate filtered samples of at least part of the image; deciding whether to perform the second stage of ALF on at least a part of the picture; when deciding not to perform the second stage ALF, avoid the second stage for the ALF Decode the filter coefficients of the one or more ALFs and avoid performing the second stage of the ALF on at least a portion of the picture; and, using the filtered samples, generate final samples for at least a portion of the picture.

，其中R̃（x,y）表示在位置（x,y）處的最終取樣之一，w _i包括針對一或多個ALF的第i個ALF的權重值，並且R'（x, y, i）包括由第i個ALF在位置（x, y）產生的經濾波取樣之一。 Clause 2B. The method of Clause 1B, wherein using the filtered samples to generate the final samples comprises: according to

, where R̃(x, y) represents one of the final samples at position (x, y), _wi includes the weight value of the ith ALF for one or more ALFs, and R'(x, y, i ) includes one of the filtered samples produced by the ith ALF at position (x, y).

Clause 3B. The method of Clause 2B, further comprising: determining the weight value w _i according to the fixed configuration data.

Clause 4B. The method of Clause 2B, further comprising: determining weight values _wi based on the decoded filter usage information.

Clause 5B. The method of Clause 2B, further comprising: decoding the weight value _wi .

Clause 6B. The method of Clause 5B, wherein decoding the weight value _wi comprises: from a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a sub-picture header, a coding tree unit ( One or more of a CTU) header, a coding tree block (CTB) header, or a sub-block header, decode the weight value _wi .

Clause 7B. The method of any of Clauses 2B-6B, further comprising: decoding data representing syntax elements to be used for the combination of weight values _wi for one or more final samples.

Clause 8B. The method of any of Clauses 1B-7B, wherein determining whether to perform the second stage of the ALF comprises decoding the material indicating whether to perform the second stage of the ALF.

Clause 9B. The method of Clause 8B, wherein decoding the material indicating whether to perform the second stage of the ALF comprises: decoding a sequence parameter set (SPS), a picture parameter set (PPS), a slice header, a sub-picture header , a Coding Tree Unit (CTU) header, a Coding Tree Block (CTB) header, or one or more of the sub-block headers for decoding.

Clause 10B. The method of any of clauses 8B and 9B, wherein decoding the material indicating whether to perform the second stage of the ALF comprises decoding a value for a syntax element indicating that the ALF is to be bypassed second stage.

Clause 11B. The method of any of clauses 1B-7B, wherein determining whether to perform the second stage of ALF comprises: self-adjustingly deriving whether to perform the second stage of ALF based on previously decoded encoding information.

Clause 12B. The method of Clause 11B, wherein the previously coded encoding information includes one or more of: a quantization parameter (QP) for a block in at least a portion of the picture, a size of a block, Or whether to use intra-frame prediction or inter-frame prediction to predict the block.

Clause 13B. The method of any of Clauses 1B-12B, wherein using the filtered samples to generate final samples comprises scaling the filtered samples to generate final samples.

Clause 14B. The method of any of Clauses 1B-13B, further comprising: encoding at least a portion of the picture prior to decoding at least the portion of the picture.

Clause 15B. An apparatus for decoding video material, the apparatus comprising one or more means for performing the method of any of Clauses 1B-14B.

Clause 16B. The apparatus of clause 15B, wherein the one or more units comprise one or more processors implemented in a circuit.

Clause 17B. The apparatus of any of clauses 15B and 16B, further comprising: a display configured to display the decoded video material.

Clause 18B. The device of any of Clauses 15B-17B, wherein the device comprises one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box. .

Clause 19B. The apparatus of clauses 15B-18B, further comprising: memory configured to store the video data.

Clause 20B. A computer-readable storage medium having instructions stored thereon that, when executed, cause a processor of an apparatus for decoding video data to perform the Methods.

Clause 21B. An apparatus for decoding video data, the apparatus comprising: means for decoding at least a portion of a picture of the video data; means for applying one or more ALFs to at least a portion of an image to generate filtered samples for at least a portion of the image; means for deciding whether to perform a second stage of the ALF on at least a portion of the picture; means for determining when A unit that avoids decoding one or more filter coefficients of the ALF in the second stage of the ALF when the second stage of the ALF is not performed; used to avoid decoding at least a portion of the picture when it is decided not to perform the second stage of the ALF means for performing a second stage of the ALF; and means for using the filtered samples to generate final samples for the at least a portion of the picture.

Clause 1C. A method of decoding video data, the method comprising: applying a first stage self-adjusting loop filter (ALF) to reconstructed samples of a reconstructed block, wherein applying the first stage ALF comprises: determining a a first class index; selecting a filter from the first set of filters based on the first class index; and applying a filter from the first set of filters to the reconstructed samples to determine the first intermediate sample value; applying the second The stage ALF is applied to the reconstructed sampling, wherein applying the second stage ALF includes: determining a second class index for the reconstructed sampling; selecting a second filter from the second set of filters based on the second class index; apply to the reconstructed samples to determine a first sample modification value; determine a second sample modification value based on the first intermediate sample value; and determine a filtered reconstruction based on the reconstructed samples, the first sample modification value and the second sample modification value sampling.

Clause 2C. The method of Clause 1C, wherein the first set of filters comprises fixed filters.

Clause 3C. The method of any of Clauses 1C-2C, wherein the second set of filters comprises: a signaled filter determined based on a syntax for signaling in the video data.

Clause 4C. The method of any of Clauses 1C-3C, wherein determining the first category index comprises: determining an activity value for reconstructing sampling; determining a direction for reconstructing sampling; and, based on the activity value and the direction Determine the first category index.

Clause 5C. The method of Clause 4C, wherein determining the direction comprises assigning one of 56 values to the direction.

Clause 6C. The method of any of Clauses 1C-5C, wherein the first set of filters comprises 9x9 diamond filters.

Clause 7C. The method of any of Clauses 1C-6C, wherein determining the filtered reconstructed samples based on the reconstructed samples, the first sample modification value, and the second sample modification value comprises: combining the first sample modification value and the second sample modification value The sample modification value is added to the reconstructed sample.

Clause 8C. The method of any of Clauses 1C-7C, wherein determining the second sample modification value based on the first intermediate sample value comprises trimming the second sample modification value to determine the trimmed sample modification value, and A sample modification value and the trimmed sample modification value are added to the reconstructed samples.

Clause 9C. The method of Clause 8C, further comprising: receiving a trimmed sample modification value in the video data.

Clause 10C. The method of any of Clauses 1C-9C, wherein determining the second sample modification value based on the first intermediate sample value comprises determining a difference between the reconstructed sample and the first intermediate sample value.

Clause 11C. The method of any of Clauses 1C-10C, further comprising: selecting the first set of filters from the complex set of fixed filters based on quantization parameters used to reconstruct the block.

Clause 12C. The method of any of Clauses 1C-11C, wherein applying the first stage ALF also comprises: determining a third class index for reconstructing samples, selecting from the second set of filters based on the third class index and, applying a third filter from the second set of filters to the reconstructed samples to determine second intermediate sample values; and applying the second stage ALF to the reconstructed samples also includes: based on the first intermediate sample values and the second intermediate sample value to determine the second sample modification value.

Clause 13C. The method of any of Clauses 1C-12C, further comprising: adding the prediction samples and the residual samples to determine the reconstructed samples.

Clause 14C. The method of any of Clause 13C, further comprising applying a deblocking filter to the sum of the prediction samples and the residual samples to determine the reconstructed samples.

Clause 15C. The method of any of clauses 1C-14C, further comprising: outputting a decoded picture of the video data, wherein the decoded picture includes filtered reconstructed samples.

Clause 16C. The method of any of clauses 1C-14C, wherein the decoding method is performed as part of a video encoding procedure.

Clause 17C. An apparatus for decoding video data, the apparatus comprising: memory configured to store video data; one or more processors implemented in circuitry and configured to: self-adjust the first stage An in-loop filter (ALF) is applied to the reconstructed samples of the reconstructed block, wherein in order to apply the first stage ALF, the one or more processors are also configured to: determine a first class index for the reconstructed samples; based on the first class index, selecting filters from the first set of filters; and applying filters from the first set of filters to the reconstructed samples to determine first intermediate sample values; applying a second stage ALF to the reconstructed samples, wherein in order to apply the second In stage ALF, the one or more processors are also configured to: determine a second class index of the reconstructed samples; select a second filter from the second set of filters based on the second class index; apply the second filter to the reconstruction sampling to determine a first sample modification value; determining a second sample modification value based on the first intermediate sample value; and determining a filtered reconstructed sample based on the reconstructed sample, the first sample modification value and the second sample modification value.

Clause 18C. The apparatus of clause 17C, wherein the first set of filters comprises fixed filters.

Clause 19C. The apparatus of any of clauses 17C-18C, wherein the second set of filters comprises: a signaled filter determined based on a syntax of the signaled in the video data.

Clause 20C. The apparatus of any of clauses 17C-19C, wherein to determine the first class index, the one or more processors are also configured to: determine an activity value for reconstruction sampling; determine for reconstruction the direction of sampling; and, determining the first class index based on the activity value and the direction.

Clause 21C. The apparatus of clause 20C, wherein to determine the direction, the one or more processors are also configured to assign one of the 56 values to the direction.

Clause 22C. The apparatus of any of clauses 17C-21C, wherein the first set of filters comprises 9x9 diamond filters.

Clause 23C. The apparatus of any of clauses 17C-22C, wherein for determining the filtered reconstructed samples based on the reconstructed samples, the first sample modification value, and the second sample modification value, the one or more processors also is configured to add the first sample modification value and the second sample modification value to the reconstructed samples.

Clause 24C. The apparatus of any of clauses 17C-23C, wherein to determine the second sampling modification value based on the first intermediate sampling value, the one or more processors are also configured to: trim the second sampling modification value to determine the trimmed sample modification value, and add the first sample modification value and the trimmed sample modification value to the reconstructed sample.

Clause 25C. The apparatus of clause 24C, wherein the one or more processors are also configured to receive trimmed sample modification values in the video data.

Clause 26C. The apparatus of any of clauses 17C-25C, wherein to determine the second sample modification value based on the first intermediate sample value, the one or more processors are also configured to determine between the reconstruction samples and the first sample modification value. A difference between intermediate sample values.

Clause 27C. The apparatus of any of clauses 17C-26C, wherein the one or more processors are also configured to: select the first set of filter blockers from the complex set of fixed filters based on the quantization parameter for reconstruction .

Clause 28C. The apparatus of any of clauses 17C-27C, wherein to apply the first stage ALF, the one or more processors are also configured to determine a third class index of reconstruction samples, based on the following conditions: The third filter class index of the third filter is selected from the set of filters, and the third filter of the second set of filters is applied to the reconstructed samples to determine the second intermediate sample value; in order to apply the second-stage ALF to the reconstructed samples, The one or more processors are also configured to determine a second sample modification value based on the first intermediate sample value and the second intermediate sample value.

Clause 29C. The apparatus of any of clauses 17C-28C, wherein the one or more processors are also configured to: add prediction samples to residual samples to determine reconstruction samples.

Clause 30C. The apparatus of clause 28C, wherein the one or more processors are also configured to: apply a deblocking filter to the sum of the prediction samples and the residual samples to determine the reconstructed samples.

Clause 31C. The apparatus of any of clauses 17C-30C, wherein the one or more processors are also configured to: output a decoded picture of the video data, wherein the decoded picture includes filtered reconstructed samples.

Clause 32C. The apparatus of any of clauses 17C-31C, wherein the apparatus comprises a video encoding apparatus.

Clause 33C. The apparatus of any of clauses 17C-32C, wherein the one or more processors are also configured to: output a decoded picture of the video data, wherein the decoded picture includes filtered reconstructed samples.

Clause 34C. The apparatus of Clause 33C, wherein the apparatus comprises a wireless communication device and also includes a receiver configured to receive encoded video data.

Clause 35C. The apparatus of clause 34C, wherein the wireless communication device comprises a telephone handset, and wherein the receiver is configured to demodulate a signal comprising encoded video material according to a wireless communication standard.

Clause 36C. The apparatus of any of Clauses 17C-35C, further comprising: a display configured to display the decoded video material.

Clause 37C. The device of any of Clauses 17C-36C, wherein the device comprises one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 38C. The apparatus of any of clauses 17C-37C, wherein the apparatus comprises a video encoding apparatus.

Clause 39C. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: apply a first-stage self-adjusting loop filter (ALF) Reconstruction samples in the reconstruction block, wherein in order to apply the first stage ALF, the one or more processors are also configured to: determine a first class index for the reconstructed samples; based on the first class index, from the first set of filters selecting a filter; and applying a filter from the first set of filters to the reconstructed samples to determine a first intermediate sample value; applying a second stage ALF to the reconstructed samples, wherein to apply the second stage ALF, one or more The processor is also configured to: determine a second class index for the reconstructed samples; select a second filter from the second set of filters based on the second class index; apply the second filter to the reconstructed samples to determine the first a sample modification value; a second sample modification value is determined based on the first intermediate sample value; and a filtered reconstructed sample is determined based on the reconstructed sample, the first sample modification value and the second sample modification value.

Clause 1D. A method of decoding video data, the method comprising: applying a first-stage self-adjusting loop filter (ALF) to reconstructed samples of a reconstruction block, wherein applying the first-stage ALF comprises: determining a a first class index; selecting a filter from the first set of filters based on the first class index; and applying a filter from the first set of filters to the reconstructed samples to determine the first intermediate sample value; applying the second stage ALF is applied to reconstructed samples, wherein applying the second-stage ALF includes: determining a second class index for reconstructing samples; selecting a second filter from a second set of filters based on the second class index; applying the second filter determining a first sample modification value based on the reconstructed samples; determining a second sample modification value based on the first intermediate sample value; and determining a filtered reconstruction based on the reconstructed samples, the first sample modification value, and the second sample modification value sampling.

Clause 2D. The method of Clause ID, wherein the first set of filters comprises fixed filters.

Clause 3D. The method of Clause ID, wherein the second set of filters comprises: signaled filters determined based on a syntax signaled in the video data.

Clause 4D. The method of Clause ID, wherein determining the first class index comprises: determining an activity value for reconstructing sampling; determining a direction for reconstructing sampling; and, based on the activity value and the direction, determining the first class index .

Clause 5D. The method of Clause 4D, wherein determining the direction comprises assigning one of 56 values to the direction.

Clause 6D. The method of Clause ID, wherein the first set of filters comprises 9x9 diamond filters.

Clause 7D. The method of Clause ID, wherein determining the filtered reconstructed sample based on the reconstructed sample, the first sample modification value, and the second sample modification value comprises adding the first sample modification value and the second sample modification value to a Rebuild sampling.

Clause 8D. The method of Clause 1D, wherein determining the second sample modification value based on the first intermediate sample value comprises trimming the second sample modification value to determine a trimmed sample modification value, and modifying the first sample modification The value and the trimmed sample modification value are added to the reconstructed sample.

Clause 9D. The method of Clause 8D, further comprising: receiving a trimmed sample modification value in the video data.

Clause 10D. The method of Clause ID, wherein determining the second sample modification value based on the first intermediate sample value comprises determining a difference between the reconstructed sample and the first intermediate sample value.

Clause 11D. The method of Clause ID, further comprising: selecting the first set of filters from the complex set of fixed filters based on quantization parameters for the reconstructed block.

Clause 12D. The method of Clause ID, wherein applying the first stage ALF also comprises: determining a third class index for reconstructing samples; selecting a second filter from the second set of filters based on the third class index; and, applying the second filter from the second set of filters to the reconstructed samples to determine the second intermediate sample value; and, applying the second stage ALF to the reconstructed sample also includes: based on the first intermediate sample value and the second intermediate sample value The sample value determines the second sample modification value.

Clause 13D. The method of Clause ID, further comprising: adding the prediction samples and the residual samples to determine the reconstruction samples.

Clause 14D. The method of Clause 13D, further comprising applying a deblocking filter to the sum of the prediction samples and the residual samples to determine the reconstructed samples.

Clause 15D. The method of Clause ID, further comprising: outputting a decoded picture of the video data, wherein the decoded picture includes filtered reconstructed samples.

Clause 16D. The method of Clause ID, wherein the decoding method is performed as part of a video encoding procedure.

Clause 17D. An apparatus for decoding video data, the apparatus comprising: memory configured to store video data; one or more processors implemented in circuitry and configured to: self-adjust the first stage An in-loop filter (ALF) is applied to the reconstructed samples of the reconstructed block, wherein in order to apply the first stage ALF, the one or more processors are also configured to: determine a first class index for the reconstructed samples; based on the first class index , selecting filters from the first set of filters; and, applying filters from the first set of filters to the reconstructed samples to determine the first intermediate sample value; applying a second-stage ALF to the reconstructed samples, wherein in order to apply the first intermediate sample value In a two-stage ALF, the one or more processors are also configured to: determine a second class index for reconstructing samples; select a second filter from a second set of filters based on the second class index; apply to reconstructed samples to determine a first sample modification value; determine a second sample modification value based on the first intermediate sample value; and determine a filtered reconstructed sample based on the reconstructed samples, the first sample modification value, and the second sample modification value .

Clause 18D. The apparatus of clause 17D, wherein the first set of filters comprises fixed filters.

Clause 19D. The apparatus of clause 17D, wherein the second set of filters comprises signaled filters determined based on a syntax of the signaled in the video data.

Clause 20D. The apparatus of clause 17D, wherein to determine the first class index, the one or more processors are also configured to: determine an activity value for reconstructing sampling; determine a direction for reconstructing sampling; and, based on Activity value and direction to determine the first category index.

Clause 21D. The apparatus of clause 20D, wherein to determine the direction, the one or more processors are also configured to assign one of the 56 values to the direction.

Clause 22D. The apparatus of Clause 17D, wherein the first set of filters comprises 9x9 diamond filters.

Clause 23D. The apparatus of clause 17D, wherein to determine the filtered reconstructed samples based on the reconstructed samples, the first sample modification value, and the second sample modification value, the one or more processors are also configured to: convert the first The sample modification value and the second sample modification value are added to the reconstructed samples.

Clause 24D. The apparatus of clause 17D, wherein to determine the second sample modification value based on the first intermediate sample value, the one or more processors are also configured to: trim the second sample modification value to determine the trimmed sample The modification value, and the first sample modification value and the trimmed sample modification value are added to the reconstructed samples.

Clause 25D. The apparatus of clause 24D, wherein the one or more processors are also configured to: receive the trimmed sample modification values in the video data.

Clause 26D. The apparatus of clause 17D, wherein to determine the second sample modification value based on the first intermediate sample value, the one or more processors are also configured to determine a difference between the reconstructed sample and the first intermediate sample value.

Clause 27D. The apparatus of clause 17D, wherein the one or more processors are further configured to: select the first set of filters from the complex set of fixed filters based on quantization parameters used to reconstruct the block.

Clause 28D. The apparatus of Clause 17D, wherein to apply the first stage ALF, the one or more processors are also configured: to determine a third class index for reconstructing the sample, from the second set of indices based on the third class index selecting a second filter among the filters, and applying a second filter from the second set of filters to the reconstructed samples to determine a second intermediate sample value; to apply the second stage ALF to the reconstructed samples, one or more The processor is also configured to determine a second sample modification value based on the first intermediate sample value and the second intermediate sample value.

Clause 29D. The apparatus of Clause 17D, wherein the one or more processors are also configured to: add the prediction samples and the residual samples to determine the reconstruction samples.

Clause 30D. The apparatus of Clause 28D, wherein the one or more processors are also configured to: apply a deblocking filter to the sum of the predicted samples and the residual samples to determine the reconstructed samples.

Clause 31D. The apparatus of clause 17D, wherein the one or more processors are also configured to: output a decoded picture of the video data, wherein the decoded picture includes filtered reconstructed samples.

Clause 32D. The apparatus of clause 17D, wherein the apparatus comprises a video encoding apparatus.

Clause 33D. The apparatus of Clause 17D, wherein the one or more processors are also configured to: output a decoded picture of the video data, wherein the decoded picture includes filtered reconstructed samples.

Clause 34D. The apparatus of Clause 33D, wherein the apparatus comprises a wireless communication device, further comprising: a receiver configured to receive the encoded video data.

Clause 35D. The apparatus of Clause 34D, wherein the wireless communication device comprises a telephone handset, and wherein the receiver is configured to demodulate the signal comprising the encoded video material according to a wireless communication standard.

Clause 36D. The apparatus of clause 17D, further comprising: a display configured to display the decoded video data.

Clause 37D. The device of Clause 17D, wherein the device comprises one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

Clause 38D. The apparatus of clause 17D, wherein the apparatus comprises a video encoding apparatus.

Clause 39D. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: apply a first-stage self-adjusting loop filter (ALF) to reconstructed samples of the reconstructed block, wherein to apply the first stage ALF, the one or more processors are also configured to: determine a first class index for the reconstructed samples; select a filter from a first set of filters based on the first class index and, applying filters from the first set of filters to the reconstructed samples to determine a first intermediate sample value; applying a second stage ALF to the reconstructed samples, wherein in order to apply the second stage ALF, the one or more The processor is also configured to: determine a second class index for the reconstructed samples; select a second filter from the second set of filters based on the second class index; apply the second filter to the reconstructed samples to determine the first a sample modification value; a second sample modification value is determined based on the first intermediate sample value; and a filtered reconstructed sample is determined based on the reconstructed sample, the first sample modification value and the second sample modification value.

It is recognized that, by way of example, certain acts or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (eg, not all described acts or events are necessary for implementing the techniques). necessary). Furthermore, in some instances, actions or events may be performed concurrently rather than sequentially, eg, via multi-threaded processing, interrupt processing, or multiple processors.

In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media, such as data storage media, or communication media, including facilitating transfer of computer programs from one place to another, for example, in accordance with communication protocols of any media. In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media such as a signal or carrier wave. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to obtain instructions, code and/or data structures for implementing the techniques described in this context. The computer program product may include a computer-readable medium.

By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or capable of Store the desired code in the form of instructions or data structures and any other medium that can be accessed by the computer. Also, any connection is properly termed a computer-readable medium. For example, if coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (eg, infrared, radio, and microwave) are used to transmit commands from a website, server, or other remote source, coaxial cable, fiber optic cable, or Fiber optic cable, twisted pair, DSL, or wireless technologies (eg, infrared, radio, and microwave) are included in the definition of media. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. As used herein, magnetic and optical discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy discs, and blu-ray discs, where magnetic discs generally reproduce data magnetically, while optical discs use lasers to reproduce data Optically reproduce material. Combinations of the above should also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or individual logic circuits. Accordingly, the terms "processor" and "processing circuit" as used herein may represent any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a combined transcoder. Furthermore, the techniques may be implemented entirely in one or more circuits or logic elements.

The techniques disclosed herein can be implemented in a wide variety of devices or devices, including a wireless handset, an integrated circuit (IC), or a set of ICs (eg, a chip set). Various components, modules, or units are described in this context to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation via distinct hardware units. Rather, various units may be combined in a transcoder hardware unit, as previously described, or by a collection of interoperable hardware units (including one or more processors as previously described) in combination with appropriate software and / or firmware provided.

Various examples have been described. These and other examples are within the scope of the appended claims.

100: System 102: Source Device 104: Video source 106: Memory 108: Output interface 110: Computer-readable media 112: Storage device 114: file server 116: destination device 118: Display device 120: memory 122: Input interface 140: Filter 142: Filter 150: Subblock 160: block 162: block 164:block 166: block 170: Grid 180: Filter 182: Filter 184: Filter 190: Filter 192:VB 200: Video Transcoder 202: Mode selection unit 204: Residual generation unit 206: Transform processing unit 208: Quantization unit 210: Inverse Quantization Unit 212: Inverse transform processing unit 214: Rebuild Unit 216: Filter unit 218: DPB 220: Entropy coding unit 222: Motion Estimation Unit 224: Motion Compensation Unit 226: In-frame prediction unit 230: Video data memory 300: Video Decoder 302: Entropy decoding unit 304: Prediction Processing Unit 306: Inverse Quantization Unit 308: Inverse transform processing unit 310: Rebuild Unit 312: Filter unit 314: DPB 316: Motion Compensation Unit 318: In-frame prediction unit 320: Coded Picture Buffer (CPB) memory 342: Deblocking Filter 344: SAO filter 346: ALF unit 400: Phase 1 ALF 410: Second Stage ALF 430: Filter sampling 510: Filter 520: Filter 530: Filter 540: Filter 542: Second Stage ALF 550: Filter 560: Filter 562: Filter 564: Filter 566: Filter 568: Filter 570: Filter 572: Filter 574: Filter 576: Filter 578: Filter 580: Phase 1 582: Stage Two 584: Phase 1 ALF 586: Second ALF Stage 588: Filter sampling 650: Square 652: Square 654: Square 656: Square 658: Square 660: Square 670: Square 672: Square 674: Square 676: Square 678: Square 680: Square 710: Blocks 712: Square 714: Square 716: Square 720: Square 722: Square 724: Square 726: Square 730: Square D1: Gradient D2: Gradient H: horizontal gradient V: vertical gradient

1 is a block diagram illustrating an example video encoding and decoding system that may implement the techniques of the present disclosure.

2A and 2B are conceptual diagrams illustrating example self-tuning loop filter shapes that may be used in accordance with the techniques disclosed in this patent.

3 is a conceptual diagram illustrating example sub-sampled Laplacian values for 4x4 sub-block self-tuning loop filter classification that may be used in accordance with the techniques disclosed herein.

4 is a conceptual diagram illustrating example Laplacian values for luma sampling that may be used in accordance with the techniques of this disclosure.

5 is a conceptual diagram illustrating an example of self-tuning loop filter class merging that may be used in accordance with the techniques of this disclosure.

6 is a conceptual diagram illustrating an example geometric transformation of a 7x7 diamond filter shape that may be used in accordance with the techniques of the present disclosure.

7A-7C illustrate an example of symmetric sampling padding in an ALF that may be used in accordance with the techniques of the present disclosure.

Figure 8 shows an example of an ALF 4x4 sub-block classification that may be used in accordance with the techniques disclosed in this case.

9 illustrates an example framework in which reconstructed samples of video data may be filtered using multiple fixed filter banks and multiple signaled filter banks in accordance with the techniques disclosed herein.

10A-10E illustrate examples of 5x5, 7x7, 9x9, 11x11, and 13x13 filters that may be used in accordance with the techniques disclosed in this case.

11 illustrates an example of multiple fixed filters that can be implemented with one signaled filter in accordance with the techniques disclosed in this application.

12A-12E illustrate examples of two-part filters that may be used in accordance with the techniques disclosed in this application.

13 and 14 are conceptual diagrams illustrating exemplary ALF frameworks that may be in accordance with techniques disclosed in this patent.

15 is a block diagram illustrating an exemplary video transcoder that may perform the techniques of the present disclosure.

16 is a block diagram illustrating an exemplary video decoder that may perform the techniques of the present disclosure.

17 is a block diagram illustrating an exemplary filter unit for implementing the techniques of this disclosure.

18 is a flowchart illustrating an exemplary procedure for encoding a current block in accordance with the techniques of this disclosure.

19 is a flowchart illustrating an exemplary procedure for decoding a current block in accordance with the techniques of this disclosure.

20 is a flowchart illustrating an exemplary procedure for decoding a current block in accordance with the techniques of this disclosure.

Domestic storage information (please note in the order of storage institution, date and number) none Foreign deposit information (please note in the order of deposit country, institution, date and number) none

540: Filter

542: Second Stage ALF

Claims (38)

A method for decoding video information, the method comprising the following steps: A first-stage self-adjusting loop filter (ALF) is applied to a reconstructed sample of a reconstruction block, wherein applying the first-stage ALF includes the following steps: determining a first class index for the reconstructed sample; selecting a filter from a first set of filters based on the first class index; and applying the filter from the first set of filters to the reconstructed samples to determine a first intermediate sample value; Applying a second-stage ALF to the reconstructed sample, wherein applying the second-stage ALF includes the following steps: determining a second class index for the reconstructed sample; selecting a second filter from a second set of filters based on the second class index; applying the second filter to the reconstructed samples to determine a first sample modification value; determining a second sample modification value based on the first intermediate sample value; and Based on the reconstructed samples, the first sample modification value, and the second sample modification value, a filtered reconstructed sample is determined. The method of claim 1, wherein the first set of filters comprises fixed filters. The method of claim 1, wherein the second set of filters includes the steps of: signaling filters determined based on a signaling syntax in the video data. The method of claim 1, wherein determining the first category index comprises the steps of: determining an activity value for the reconstructed sampling; determine a direction to use for the reconstruction sampling; and Based on the activity value and the direction, the first category index is determined. The method of claim 4, wherein determining the direction comprises assigning one of 56 values to the direction. The method of claim 1, wherein the first set of filters includes a 9x9 diamond filter. The method of claim 1, wherein determining the filtered reconstructed sample based on the reconstructed sample, the first sample modification value and the second sample modification value comprises the steps of: the first sample modification value and the second sample Modified values are added to the reconstructed sample. The method of claim 1, wherein determining the second sample modification value based on the first intermediate sample value comprises the steps of trimming the second sample modification value to determine a trimmed sample modification value, and modifying the first sample modification The value and the trimmed sample modification value are added to the reconstructed sample. According to the method of claim 8, it also includes the following steps: The trimmed sample modification value is received in the video data. The method of claim 1, wherein determining the second sample modification value based on the first intermediate sample value comprises the step of: determining a difference between the reconstructed sample and the first intermediate sample value. According to the method of claim 1, the following steps are also included: The first set of filters is selected from the complex set of fixed filters based on a quantization parameter for the reconstructed block. The method according to claim 1, wherein Applying the first-stage ALF also includes the steps of: determining a third class index for the reconstructed sample, selecting a third filter from a second set of filters based on the third class index, and converting the The third filter of the two sets of filters is applied to the reconstructed samples to determine a second intermediate sample value; Applying the second-stage ALF to the reconstructed sample also includes determining the second sample modification value based on the first intermediate sample value and the second intermediate sample value. According to the method of claim 1, the following steps are also included: A prediction sample is added to a residual sample to determine the reconstructed sample. According to the method of claim 13, the following steps are also included: A deblocking filter is applied to the sum of the prediction samples and the residual samples to determine the reconstructed samples. According to the method of claim 1, the following steps are also included: A decoded picture of the video data is output, wherein the decoded picture includes the filtered reconstructed samples. The method of claim 1, wherein the method of decoding is performed as part of a video encoding process. A device for decoding video information, the device includes: a memory configured to store video data; one or more processors implemented in circuitry and configured to: Applying a first-stage self-adjusting loop filter (ALF) to a reconstructed sample of a reconstruction block, wherein to apply the first-stage ALF, the one or more processors are also configured to: determining a first class index for the reconstructed sample; selecting a filter from a first set of filters based on the first class index; and applying the filter from the first set of filters to the reconstructed samples to determine a first intermediate sample value; Applying a second-stage ALF to the reconstructed samples, wherein to apply the second-stage ALF, the one or more processors are also configured to: determining a second class index for the reconstructed sample; selecting a second filter from a second set of filters based on the second class index; applying the second filter to the reconstructed samples to determine a first sample modification value; determining a second sample modification value based on the first intermediate sample value; and Based on the reconstructed samples, the first sample modification value, and the second sample modification value, a filtered reconstructed sample is determined. The apparatus of claim 17, wherein the first set of filters comprises fixed filters. The apparatus of claim 17, wherein the second set of filters includes signaled filters determined based on a syntax of the signaled in the video data. The apparatus of claim 17, wherein to determine the first class index, the one or more processors are also configured to: determining an activity value for the reconstructed sampling; determine a direction to use for the reconstruction sampling; and Based on the activity value and the direction, the first category index is determined. The apparatus of claim 20, wherein in order to determine the direction, the one or more processors are also configured to assign one of 56 values to the direction. The apparatus of claim 17, wherein the first set of filters includes a 9x9 diamond filter. The apparatus of claim 17, wherein in order to determine the filtered reconstructed samples based on the reconstructed samples, the first sample modification value, and the second sample modification value, the one or more processors are also configured to: The first sample modification value and the second sample modification value are added to the reconstructed sample. The apparatus of claim 17, wherein to determine the second sample modification value based on the first intermediate sample value, the one or more processors are also configured to: trim the second sample modification value to determine a trimmed sample modification value value, and add the first sample modification value and the trimmed sample modification value to the reconstructed sample. The apparatus of claim 24, wherein the one or more processors are also configured to: The trimmed sample modification value is received in the video data. The apparatus of claim 17, wherein in order to determine the second sample modification value based on the first intermediate sample value, the one or more processors are also configured to determine a difference between the reconstructed sample and the first intermediate sample value One bad. The apparatus of claim 17, wherein the one or more processors are also configured to: The first set of filters is selected from the complex set of fixed filters based on a quantization parameter for the reconstructed block. Apparatus according to claim 17, wherein To apply the first-stage ALF, the one or more processors are also configured to: determine a third class index for the reconstructed sample, select a filter from a second set of filters based on the third class index a third filter, and applying the third filter from the second set of filters to the reconstructed samples to determine a second intermediate sample value; To apply the second stage ALF to the reconstructed samples, the one or more processors are also configured to determine the second sample modification value based on the first intermediate sample value and the second intermediate sample value. The apparatus of claim 17, wherein the one or more processors are also configured to: A prediction sample is added to a residual sample to determine the reconstructed sample. The apparatus of claim 29, wherein the one or more processors are also configured to: A deblocking filter is applied to one of the predicted samples and the residual samples to determine the reconstructed samples. The apparatus of claim 17, wherein the one or more processors are also configured to: A decoded picture of the video data is output, wherein the decoded picture includes filtered reconstructed samples. The apparatus of claim 17, wherein the apparatus comprises a video encoding apparatus. The apparatus of claim 32, wherein the apparatus comprises a wireless communication device, further comprising the step of: a receiver configured to receive the encoded video data. The apparatus of claim 33, wherein the wireless communication device comprises a telephone handset, and wherein the receiver is configured to demodulate a signal comprising the encoded video data according to a wireless communication standard. Equipment according to claim 17, also including: A display configured to display the decoded video data. An apparatus according to claim 17, wherein the apparatus comprises one or more of the following: a camera, a computer, a mobile device, a broadcast receiver device or a set-top box. The apparatus of claim 17, wherein the apparatus comprises a video encoding apparatus. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: Applying a first-stage self-adjusting loop filter (ALF) to a reconstructed sample of a reconstruction block, wherein to apply the first-stage ALF, the one or more processors are also configured to: determining a first class index for the reconstructed sample; selecting a filter from a first set of filters based on the first class index; and applying the filter from the first set of filters to the reconstructed samples to determine a first intermediate sample value; Applying a second-stage ALF to the reconstructed samples, wherein to apply the second-stage ALF, the one or more processors are also configured to: determining a second class index for the reconstructed sample; selecting a second filter from a second set of filters based on the second class index; applying the second filter to the reconstructed samples to determine a first sample modification value; determining a second sample modification value based on the first intermediate sample value; and Based on the reconstructed samples, the first sample modification value, and the second sample modification value, a filtered reconstructed sample is determined.

Images