RU2795695C2 - Deblocking filter for video encoding and processing - Google Patents

Abstract

FIELD: video encoding and decoding.

SUBSTANCE: video decoder is proposed that is configured to receive the first reconstructed block of video data that includes the first discrete sample with the first value; deblocking filtering is applied to the first reconstructed block to create a deblocked filtered block; the first cut-off value is determined for the first discrete sample based on location of the first discrete sample relative to the boundary of the reconstructed block and based on the size of the reconstructed block; the first clipping value is compared with the amount of change of the first discrete sample due to deblocking filtering; in response to the change value of the first discrete sample due to deblocking filtering greater than the first clip value, change the first value using the first clip value to determine the first filtered value for the first sample, and output the deblocked filtered video data block with the first sample having the first filtered value.

EFFECT: reduction of blocking artifacts along the edges of blocks, which arise as a result of the video data recovery process.

44 cl, 11 dwg, 1 tbl

Description

[0001] The present application claims priority according to:

U.S. Patent Application No. 16/551,388, filed August 26, 2019, which claims priority:

U.S. Provisional Application No. 62/723,408, filed August 27, 2018; And

U.S. Provisional Application No. 62/742,331, filed October 6, 2018,

the entire contents of which are incorporated herein by reference.

FIELD OF TECHNOLOGY

[0002] The present disclosure relates to video coding and video decoding.

PRIOR ART

[0003] Digital video functionality can be incorporated into a wide range of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-readers, digital cameras, digital recorders, digital media players, video game devices, video game consoles, cellular or satellite radio telephones, so-called “smartphones”, video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4 Part 10, Advanced Video Coding (AVC ), High Efficiency Video Coding (HEVC), ITU-T H.265/High Efficiency Video Coding (HEVC), and extensions to such standards. Video devices can transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video coding techniques.

[0004] Video coding techniques include spatial (intra-picture, intra-picture) prediction and/or temporal (inter-picture, inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video picture or a portion of a video picture) may be broken down into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and/or coding nodes. . Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in adjacent blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction with respect to reference samples in adjacent blocks in the same picture, or temporal prediction with respect to reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames.

Brief description of the essence of the invention

[0005] The present disclosure describes techniques related to filtering recovered video data in a video encoding and/or video decoding process, and more specifically, the present disclosure describes techniques related to techniques for performing deblocking filtering. Deblocking filtering is a type of filtering that can reduce deblocking artifacts along block edges that result from the deblocking process.

[0006] According to an example, a method for decoding video data includes obtaining a first reconstructed block of video data, the first reconstructed block including a first sample with a first value; applying deblocking filtering to the first reconstructed block to create a deblocked filtered block; determining a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; comparing the first clipping value with the amount of change of the first discrete sample due to deblocking filtering; in response to a change amount of the first sample due to the deblocking filtering greater than the first clip value, changing the first value by the first clip value to determine the first filtered value for the first sample; and outputting the deblocked filtered video data block, wherein in the deblocked filtered block, the first sample has a first filtered value.

[0007] In accordance with another example, a device for decoding video data includes a memory adapted to store video data and one or more processors implemented in circuits and configured to receive a first reconstructed block of video data, the first reconstructed block including itself the first discrete sample with the first value; apply deblocking filtering to the first reconstructed block to create a deblocked filtered block; determine a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; compare the first clipping value with the amount of change of the first discrete sample due to deblocking filtering; in response to a change amount of the first sample due to the deblocking filtering greater than the first cutoff value, modifying the first value by the first cutoff value to determine a first filtered value for the first sample; and outputting the deblocked filtered video data block, wherein in the deblocked filtered block, the first sample sample has the first filtered value.

[0008] In accordance with another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause one or more processors to obtain a first recovered block of video data, the first recovered block including a first discrete sample with a first value ; apply deblocking filtering to the first reconstructed block to create a deblocked filtered block; determine a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; compare the first clipping value with the amount of change of the first discrete sample due to deblocking filtering; in response to a change amount of the first sample due to the deblocking filtering greater than the first cutoff value, modifying the first value by the first cutoff value to determine a first filtered value for the first sample; and outputting the deblocked filtered block of video data, wherein in the deblocked filtered block, the first sample sample has the first filtered value.

[0009] According to another example, an apparatus for decoding video data includes means for obtaining a first reconstructed block of video data, the first reconstructed block including a first sample with a first value; means for applying deblocking filtering to the first reconstructed block to create a deblocking filtered block; means for determining a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; means for comparing the first clipping value with a change amount of the first sample sample due to deblocking filtering; means for changing the first value by the first clipping value for determining a first filtered value for the first sample in response to a change amount of the first sample sample due to deblocking filtering greater than the first clipping value; and means for outputting the deblocked filtered video data block, wherein in the deblocked filtered block, the first sample has a first filtered value.

[0010] Details of one or more examples are provided in the accompanying drawings and in the description below. Other features, objectives and advantages will be apparent from the description, drawings and claims.

Brief description of the drawings

[0011] FIG. 1 is a block diagram illustrating an exemplary video encoding and decoding system that can perform the techniques of the present disclosure.

[0012] FIG. 2A and 2B are conceptual diagrams illustrating an exemplary quadtree-binary tree (QTBT) structure and corresponding coding tree unit (CTU).

[0013] FIG. 3 shows an example of a four-sample segment to be deblocked with adjacent P and Q blocks.

[0014] FIG. 4 shows an exemplary video data block where the second derivatives of the four circled discrete samples are calculated, the sum of which is used to determine whether to apply deblocking on the segment.

[0015] FIG. 5 shows an example video block samples for one of the four lines in the segment to be deblocked.

[0016] FIG. 6 is a block diagram illustrating an exemplary video encoder that may perform the techniques of the present disclosure.

[0017] FIG. 7 is a block diagram illustrating an exemplary video decoder that can perform the techniques of the present disclosure.

[0018] FIG. 8 is a block diagram illustrating an exemplary filter module for performing the techniques of the present disclosure.

[0019] FIG. 9 is a flowchart illustrating an example of a video encoding process.

[0020] FIG. 10 is a flowchart illustrating an example of a video decoding process.

[0021] FIG. 11 is a flowchart illustrating an example of a video decoding process.

DETAILED DESCRIPTION

[0022] Video encoding (e.g., video encoding and/or video decoding) typically involves predicting a video data block from an already coded video data block in the same picture (e.g., intra-prediction) or an already coded video data block in another picture (for example, inter-prediction). In some cases, the video encoder also calculates residual data by comparing the prediction block with the original block. Thus, the residual data represents the difference between the prediction block and the original block. To reduce the number of bits required for signaling the residual data, the video encoder transforms and quantizes the residual data and signals the transformed and quantized residual data in the encoded bitstream. The compression provided by the transformation and quantization processes may be lossy, which means that the transformation and quantization processes may introduce distortion into the video data being decoded.

[0023] The video decoder decodes and adds the residual data to the prediction block to form a reconstructed video block that matches the original video block more closely than the prediction block alone. Due to the losses introduced by the transformation and quantization of the residual data, the first reconstructed block may have distortion or artifacts. One common type of artifact or distortion is referred to as blockiness, where the boundaries of the blocks used to encode video data are visible.

[0024] To further improve the quality of the decoded video, the video decoder may perform one or more filtering operations on the reconstructed video blocks. Examples of these filtering operations include deblocking filtering, adaptive sample bias (SAO) filtering, and adaptive loop filtering (ALF). The parameters for these filtering operations may either be determined by the video encoder and explicitly signaled in the encoded video bitstream, or may be implicitly determined by the video decoder without requiring explicit signaling of the parameters in the encoded video bitstream.

[0025] The present disclosure describes techniques associated with filtering recovered video data in a video encoding and/or video decoding process, and more specifically, the present disclosure describes techniques related to deblocking filtering. Deblocking filtering is a type of filtering specifically designed for deblocking. This disclosure describes techniques related to the filtering process performed on video frames or pictures, such as video frames or pictures that may be distorted due to compression, blurring, etc., but these techniques should not be construed as being limited to the above. examples of distortion. These techniques can improve the objective and subjective quality of the video. The techniques described in this document may be used in the design of new video coding solutions such as H.266 or to extend any of the existing video codecs such as H.265/High Efficiency Video Coding (HEVC) or may be proposed as promising encoding tool for future video encoding standards. The described techniques can also be used as a post-processing method on video frames output from standard or proprietary codecs.

[0026] Current versions of HEVC use a maximum block size of 64x64. However, future video coding standards, such as the multipurpose video coding standard under development, may use block sizes larger than 64x64. The present disclosure describes techniques that can improve the distortion reduction obtained by deblocking filtering, in particular in connection with increased block sizes. In addition, the techniques described herein can achieve this improved distortion reduction while maintaining desired levels of computational complexity and coding efficiency.

[0027] As used in this disclosure, the term "video coding" generally refers to either video coding or video decoding. Likewise, the term “video encoding device” may refer generally to a video encoder or video decoder. Moreover, some of the techniques described in this disclosure in relation to video decoding may also be applied to video coding, and vice versa. For example, video encoders and video decoders are often configured to perform the same process or reciprocal processes. Also, a video encoder typically performs video decoding (also referred to as decoding) as part of the processes of determining how to encode video data. For example, the video encoder may perform deblocking filtering on the decoded video blocks to determine if some coding scheme provides the desired rate-distortion trade-off, and the video encoder may perform motion estimation using the same blocks available to the video decoder when the video decoder performs motion compensation.

[0028] FIG. 1 is a block diagram illustrating an exemplary video encoding and decoding system 100 that can perform the deblocking filtering techniques of the present disclosure. The techniques of the present disclosure are generally directed to encoding (encoding and/or decoding) video data, and more specifically, deblocking filtering techniques for video data. In principle, video data includes any video processing data. Thus, the video data may include raw, unencoded video, encoded video, decoded (eg, reconstructed) video, and video metadata such as signaling data.

[0029] As shown in FIG. 1, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by destination device 116 (destination), in this example. In particular, source device 102 provides video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can be any of a wide variety of devices, including desktop computers, laptops, tablets, set-top boxes, phones, such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, and the like. In some cases, source device 102 and destination device 116 may be equipped for wireless communication and thus may be referred to as wireless communication devices.

[0030] In the example of FIG. 1, source device 102 includes video source 104, memory 106, video encoder 200, and output interface 108. Destination device 116 includes input interface 122, video decoder 300, memory 120, and display device 118. In accordance with the present disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 may be configured to apply the deblocking filtering techniques described in this disclosure. Thus, the source device 102 is an example of a video encoding device, while the destination device 116 is an example of a video decoding device. In other examples, the source device and the destination device may include other components or devices. For example, source device 102 may receive video data from an external video source such as an external camera. Likewise, destination device 116 may communicate with an external display device instead of including an embedded display device.

[0031] System 100, as shown in FIG. 1 is just one example. In principle, any digital video encoding and/or decoding device can perform the deblocking filtering techniques described in this disclosure. Source device 102 and destination device 116 are just examples of such encoders in which source device 102 generates encoded video data for transmission to destination device 116 . The present disclosure refers to an “encoder” as a device that performs encoding (encoding and/or decoding) of data. Thus, video encoder 200 and video decoder 300 are examples of encoders, specifically a video encoder and a video decoder, respectively. In some examples, source device 102 and destination device 116 may operate substantially symmetrically such that source device 102 and destination device 116 each include video encoding and decoding components. Therefore, system 100 can support unidirectional or bidirectional video transmission between source device 102 and destination device 116, such as for video streaming, video playback, video broadcasting, or video telephony.

[0032] In general, video source 104 represents a source of video data (i.e., raw, unencoded video data) and provides a sequence of pictures (also referred to as “frames”) of video data to video encoder 200, which encodes the data for the pictures. Video source 104 of source device 102 may include a video capture device such as a video camera, a video archive containing previously captured raw video, and/or a video feed interface for receiving video from a video content provider. As a further alternative, video source 104 may generate computer graphics based data as source video or a combination of live video, archived video, and computer generated video. In each case, video encoder 200 encodes captured, pre-captured, or computer-generated video data. The video encoder 200 may reorder the pictures from the received order (sometimes referred to as "display order") into the encoding order for encoding. Video encoder 200 may generate a bitstream including encoded video data. The source device 102 may then output the encoded video data via the output interface 108 to the computer-readable medium 110 for receiving and/or retrieving the data, for example, by the input interface 122 of the destination device 116.

[0033] The memory 106 of the source device 102 and the memory 120 of the destination device 116 are general purpose memory. In some examples, memories 106 and 120 may store raw video data, such as raw video from video source 104 and raw decoded video data from video decoder 300. Additionally or alternatively, memories 106 and 120 may store program instructions executable by, for example, video encoder 200 and video decoder 300, respectively. Although memories 106 and 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include internal memories for functionally similar or equivalent purposes. In addition, memories 106 and 120 may store encoded video data, such as output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106 and 120 may be allocated as one or more video buffers, such as for storing raw, decoded, and/or encoded video data.

[0034] Computer readable medium 110 may be any type of medium or device capable of transporting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium allowing source device 102 to transmit encoded video data directly to destination device 116 in real time, such as via an RF network or computer network. The output interface 108 may modulate a transmission signal including encoded video data, and the input interface 122 may demodulate the received transmission signal in accordance with a communication standard such as a wireless communication protocol. The communication medium may comprise one or both of a wireless or wired communication medium such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet network such as a local area network, a wide area network, or a wide area network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from source device 102 to destination device 116.

[0035] In some examples, computer-readable medium 110 may include a storage device 112. Source device 102 may output encoded data from output interface 108 to storage device 112 . Likewise, destination device 116 may access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally available storage media such as a hard disk drive, Blu-ray discs. , DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data.

[0036] In some examples, computer-readable medium 110 may include a file server 114 or other intermediate storage device that may store encoded video data generated by source device 102. The source device 102 may output the encoded video data to the file server 114. The destination device 116 may access the stored video data from the file server 114 via streaming or download. File server 114 may be any type of server device capable of storing encoded video data and transmitting this encoded video data to destination device 116 . The file server 114 may be a web server (eg, for a website), a file transfer protocol (FTP) server, a network content delivery device, or a network attached storage (NAS) device. The destination device 116 can access the encoded video data from the file server 114 via any standard data connection, including an Internet connection. This may include a wireless link (such as a Wi-Fi connection), a wired connection (such as a digital subscriber line (DSL), cable modem, etc.) or a combination of both, which is suitable for accessing encoded video data stored on file server 114. File server 114 and input interface 122 may be configured to operate in accordance with a streaming protocol, a download transfer protocol, or a combination thereof.

[0037] Output interface 108 and input interface 122 may be wireless transmitters/receivers, modems, wired network components (eg, Ethernet cards), wireless communications components operating in accordance with any of a variety of IEEE 802.11 standards, or other physical components. . In examples where output interface 108 and input interface 122 contain wireless components, output interface 108 and input interface 122 may be configured to transmit data such as encoded video data in accordance with a cellular standard such as 4G, 4G-LTE (Long-term development), LTE-Advanced (Advanced LTE), 5G or the like. In some examples, where the output interface 108 comprises a wireless transmitter, the output interface 108 and the input interface 122 may be configured to transmit data such as encoded video data in accordance with other wireless standards such as the IEEE 802.11 specification, the IEEE 802.15 specification ( such as ZigBee™), the Bluetooth™ standard, or the like. In some examples, source device 102 and/or destination device 116 may include respective single-chip (SoC) devices. For example, source device 102 may include an SoC device to perform functionality related to video encoder 200 and/or output interface 108, and destination device 116 may include an SoC device to perform functionality related to video decoder 300 and /or input interface 122.

[0038] The techniques of the present disclosure may be applied to video coding supporting any of a variety of multimedia applications such as terrestrial television transmissions, cable television transmissions, satellite television transmissions, Internet video streaming such as Dynamic Adaptive Streaming over HTTP (DASH ), digital video that is encoded on a storage medium, decoding digital video stored on a storage medium, or other applications.

[0039] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (eg, from a communication medium, a storage device 112, a file server 114, or the like). The encoded video bitstream from computer-readable medium 110 may include signaling information defined by video encoder 200 that is also used by video decoder 300, such as syntax elements having values that describe characteristics and/or processing of video blocks or other coded units (e.g., , clippings, pictures, groups of pictures, sequences, or the like). The display device 118 displays decoded pictures of the decoded video data to the user. Display device 118 may be any of a variety of display devices such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light emitting diode (OLED) display, or other type of display device.

[0040] Although not shown in FIG. 1, in some examples, video encoder 200 and video decoder 300 may be integrated with an audio encoder and/or audio decoder and may include appropriate mux-demux (MUX-DEMUX) modules or other hardware and/or software to handle multiplexed streams. , including both audio and video, in a common data stream. If applicable, MUX-DEMUX modules may conform to the ITU H.223 multiplexer protocol or other protocols such as User Datagram Protocol (UDP).

[0041] Both video encoder 200 and video decoder 300 may be implemented as any of a variety of suitable encoder and/or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are partially implemented in software, a device may store the instructions for the software on a suitable, non-volatile computer-readable medium and execute the instructions in hardware using one or more processors to execute the techniques of the present disclosure. Each of video encoder 200 and video decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder/decoder (CODEC) in a respective device. The device including the video encoder 200 and/or video decoder 300 may include an integrated circuit, a microprocessor, and/or a wireless communication device such as a cellular phone.

[0042] The video encoder 200 and video decoder 300 may operate in accordance with a video coding standard such as ITU-T H. 265, also referred to as High Efficiency Video Coding (HEVC), or extensions thereof, such as multi-view and/or scalable video coding extensions. video encoding. In other examples, video encoder 200 and video decoder 300 may operate in accordance with other proprietary or industry standards such as the Joint Exploration Test Model (JEM) or ITU-T H.266, also referred to as Multipurpose Video Coding. (Versatile Video Coding, VVC). A recent draft VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, ^15th Meeting: Gothenburg, SE, 3-12 July 2019, JVET-O2001-vE (hereinafter referred to as “VVC Draft 6”). However, the techniques of the present disclosure are not limited to any particular coding standard.

[0043] In general, video encoder 200 and video decoder 300 may perform block coding of pictures. The term “block” generally refers to a structure that includes data to be processed (eg, encoded, decoded, or otherwise used in an encoding and/or decoding process). For example, the block may include a two-dimensional matrix of samples (samples) of luma and/or chrominance data. In general, video encoder 200 and video decoder 300 may encode video data represented in YUV format (eg, Y, Cb, Cr). That is, instead of encoding red, green, and blue (RGB) data for the picture samples, video encoder 200 and video decoder 300 may encode luminance and chrominance components, where the chrominance components may include both red tint and blue tint chrominance components. In some examples, video encoder 200 converts the received RGB data to a YUV representation before encoding, and video decoder 300 converts the YUV representation to RGB. Alternatively, these transformations may be performed by pre- and post-processing modules (not shown).

[0044] The present disclosure may generally refer to encoding (eg, encoding and decoding) of pictures, including the process of encoding or decoding picture data. Likewise, the present disclosure may refer to picture block coding, including the process of encoding or decoding data for blocks, such as prediction and/or residual coding. The encoded video bitstream typically includes a sequence of values for syntax elements representing encoding decisions (eg, encoding modes) and blocking pictures. Thus, references to encoding a picture or block should generally be understood as encoding values for the syntax elements that make up the picture or block.

[0045] HEVC defines various blocks, including coding units (CU), prediction units (PU), and transform units (TU). According to HEVC, a video encoder (such as video encoder 200) partitions a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video encoder splits the CTU and CU into four equal, non-overlapping squares, and each quadtree node has either zero or four child nodes. Nodes with no children may be referred to as "leaf nodes" and the CUs of such leaf nodes may include one or more PUs and/or one or more TUs. The video encoder may further split the PU and TU. For example, in HEVC, the residual quadtree (RQT) is a partition of the TU. In HEVC, PUs represent inter-prediction data while TUs represent residual data. CUs that are intra predicted include intra prediction information such as an intra mode indication.

[0046] The video encoder 200 and video decoder 300 may operate in accordance with a video coding standard such as ITU-T H. 265, also referred to as High Efficiency Video Coding (HEVC), or extensions thereof, such as multi-view and/or scalable video coding extensions. video encoding. Alternatively, video encoder 200 and video decoder 300 may operate in accordance with other proprietary or industry standards such as the Joint Exploration Test Model (JEM) or ITU-T H.266, also referred to as Versatile Video Coding (Versatile Video Coding, VVC). A recent draft VVC standard is described in Bross, et al. “Versatile Video Coding (Draft 6),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, ^15th Meeting: Gothenburg, SE, 3-12 July 2019, JVET-O2001-vE (hereinafter referred to as “VVC Draft 6”). However, the techniques of the present disclosure are not limited to any particular coding standard.

[0047] In general, video encoder 200 and video decoder 300 may perform block coding of pictures. The term “block” generally refers to a structure containing data to be processed (eg, encoded, decoded, or otherwise used in the encoding and/or decoding process). For example, the block may include a two-dimensional matrix of luma and/or chrominance data samples. In general, video encoder 200 and video decoder 300 may encode video data represented in YUV format (eg, Y, Cb, Cr). That is, instead of encoding red, green, and blue (RGB) data for the picture samples, video encoder 200 and video decoder 300 may encode luminance and chrominance components, where the chrominance components may include both red tint and blue tint chrominance components. In some examples, video encoder 200 converts the received RGB data to a YUV representation prior to encoding, and video decoder 300 converts the YUV representation to RGB. Alternatively, these transformations may be performed by pre- and post-processing modules (not shown).

[0048] The present disclosure may generally refer to encoding (eg, encoding and decoding) of pictures, including the process of encoding or decoding picture data. Likewise, the present disclosure may refer to picture block coding, including the process of encoding or decoding data for blocks, such as prediction and/or residual coding. The encoded video bitstream typically includes a sequence of values for syntax elements representing coding decisions (eg, coding modes) and picture blocking. Thus, references to encoding a picture or block should generally be understood as encoding values for the syntax elements that make up the picture or block.

[0049] HEVC defines various blocks, including coding units (CU), prediction units (PU), and transform units (TU). According to HEVC, a video encoder (such as video encoder 200) partitions a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video encoder splits the CTU and CU into four equal, non-overlapping squares, and each quadtree node has either zero or four child nodes. Nodes without child nodes may be referred to as "leaf nodes" and the CUs of such leaf nodes may include one or more PUs and/or one or more TUs. The video encoder may further split the PU and TU. For example, in HEVC, the residual quadtree (RQT) is a partition of the TU. In HEVC, PUs represent inter-prediction data while TUs represent residual data. CUs that are intra predicted include intra prediction information such as an intra mode indication.

[0050] As another example, video encoder 200 and video decoder 300 may be configured to operate in accordance with JEM or VVC. According to JEM or VVC, a video encoder (such as video encoder 200) splits a picture into multiple coding tree units (CTUs). The video encoder 200 may partition the CTU according to a tree structure, such as a quadtree-binary tree (QTBT) structure or a multiple type tree (MTT) structure. The QTBT structure eliminates the concept of multiple types of partitioning, such as partitioning between CUs, PUs, and TUs in HEVC. The QTBT structure may include two levels: a first level divided according to a quadtree split and a second level divided according to a binary tree split. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of binary trees correspond to coding units (CUs).

[0051] In an MTT split structure, blocks can be partitioned using a quadtree split (QT), a binary tree split (BT), and one or more types of ternary tree (TT) splits (also referred to as a ternary tree (TT)). A ternary or ternary tree split is a split where a block is split into three subblocks. In some examples, splitting a ternary or ternary tree divides the block into three subblocks without dividing the original block through the center. Partition types in MTT (for example, QT, BT, and TT) can be symmetric or asymmetric.

[0052] In some examples, video encoder 200 and video decoder 300 may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, while in other examples, video encoder 200 and video decoder 300 may use two or more QTBT or MTT structures, eg one QTBT/MTT structure for the luma component and another QTBT/MTT structure for both chrominance components (or two QTBT/MTT structures for the respective chrominance components).

[0053] Video encoder 200 and video decoder 300 may be configured to use HEVC quadtree splitting, QTBT splitting, MTT splitting, or other splitting structures. For purposes of explanation, a description of the techniques of the present disclosure is provided in relation to splitting QTBT. However, it should be understood that the techniques of the present disclosure can also be applied to video encoders configured to use quadtree partitioning or other types of partitioning.

[0054] Units (eg, CTU or CU) can be grouped in various ways in a picture. As one example, a “bar” (block) may refer to a rectangular area of CTU rows within a particular rectangular tile in a picture. A tile may be a rectangular area of the CTU within a particular column of tiles and a particular row of tiles in a picture. A tile column refers to a rectangular area of the CTU having a height equal to the height of the picture and a width given by syntax elements (eg, such as in a picture parameter set). A tile row refers to a rectangular area of the CTU having a height given by syntax elements (eg, such as in a picture parameter set) and a width equal to the width of the picture.

[0055] In some examples, a tile may be broken into a plurality of bars, each of which may include one or more CTU lines within the tile. A tile that is not broken into multiple bars may also be referred to as a bar. However, a bar that is a true subset of a tile cannot be referred to as a tile.

[0056] The bars in the picture may be arranged in a cutout. A slice can be an integer number of picture bars that can exclusively be contained to one Network Abstraction Layer (NAL) unit. In some examples, the cut includes either a series of complete tiles, or only a rigid sequence of full bars of a single tile.

[0051] The present disclosure may use “N×N” and “N by N” interchangeably to refer to sample sizes of a block (such as a CU or other video block) in terms of vertical and horizontal dimensions, such as 16×16 samples or samples 16 by 16. In general, a 16×16 CU will have 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an N×N CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in rows and columns. In addition, CUs do not need to have the same number of samples in the horizontal direction and in the vertical direction. For example, a CU may contain N×M samples, where M is not necessarily equal to N.

[0058] The video encoder 200 encodes video data for CUs representing prediction information and/or residual information and other information. The prediction information indicates how the CU should be predicted in order to generate a prediction block for the CU. The residual information is typically the sample-to-sample differences between pre-coding CU samples and the prediction block.

[0059] For CU prediction, video encoder 200 may typically generate a prediction block for the CU via inter-prediction or intra-prediction. Inter prediction generally refers to CU prediction from previously coded picture data, while intra prediction generally refers to CU prediction from previously coded data of the same picture. To perform inter-prediction, video encoder 200 may generate a prediction block using one or more motion vectors. Video encoder 200 can typically perform a motion search to identify a reference block that closely matches the CU, eg, in terms of differences between the CU and the reference block. Video encoder 200 may compute a difference metric using sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), root mean square differences (MSD), or other similar difference calculations to determine whether a reference block is close to consistent with the current CU. In some examples, video encoder 200 may predict the current CU using unidirectional prediction or bidirectional prediction.

[0060] Some examples of JEM and VVC also provide an affine motion compensation mode, which can be considered an inter-prediction mode. In affine motion compensation mode, video encoder 200 may determine two or more motion vectors that represent non-translational motion such as zoom in or out, rotation, perspective movement, or other irregular types of motion.

[0061] To perform intra prediction, video encoder 200 may select an intra prediction mode to generate a prediction block. Some Examples JEM and VVC provide sixty-seven intra-prediction modes, including various directional modes as well as planar and DC modes. In general, video encoder 200 selects an intra-prediction mode that describes adjacent samples for the current block (eg, CU) from which the current block samples should be predicted. Such samples may typically be above, above and to the left or left of the current block in the same picture as the current block, assuming that video encoder 200 encodes CTUs and CUs in raster scan order (left to right, top to bottom).

[0062] The video encoder 200 encodes data representing the prediction mode for the current block. For example, for inter prediction modes, video encoder 200 may encode data representing which of the various available inter prediction modes is in use, as well as motion information for the corresponding mode. For example, for unidirectional or bidirectional inter-prediction, video encoder 200 may encode motion vectors using advanced motion vector prediction (AMVP) or merge mode. Video encoder 200 may use similar modes to encode motion vectors for affine motion compensation mode.

[0063] After prediction, such as intra-prediction or inter-prediction of a block, video encoder 200 may calculate residual data for the block. Residual data, such as a residual block, represents the sample-to-sample differences between a given block and a prediction block for a given block generated using the corresponding prediction mode. Video encoder 200 may apply one or more transforms to the residual block to obtain transformed data in the transform region instead of the sample region. For example, video encoder 200 may apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the video residual data. In addition, the video encoder 200 may apply a secondary transform after the first transform, such as mode-dependent indivisible secondary transform (MDNSST), signal-dependent transform, Karhunen-Loeve transform (KLT), and the like. Video encoder 200 generates transform coefficients after applying one or more transforms.

[0064] As noted above, after any transformations to obtain transform coefficients, video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to a process in which transform coefficients are quantized to reduce the amount of data used to represent the coefficients as much as possible, allowing for further compression. By performing a quantization process, video encoder 200 may reduce the bit depth associated with some or all of the coefficients. For example, video encoder 200 may round an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, video encoder 200 may perform a bitwise right shift of the value to be quantized.

[0065] After quantization, video encoder 200 may scan the transform coefficients, forming a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place the higher energy (and therefore lower frequency) transform coefficients at the beginning of the vector, and to place the lower energy (and hence higher frequency) transform coefficients at the end of the vector. In some examples, video encoder 200 may use a predefined scan order to scan the quantized transform coefficients to obtain a serialized vector and then entropy encode the quantized transform coefficients of the vector. In other examples, video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to generate a one-dimensional vector, video encoder 200 may entropy encode the one-dimensional vector, such as in accordance with context adaptive binary arithmetic coding (CABAC). Video encoder 200 may also entropy encode values for syntax elements describing metadata associated with encoded video data for use by video decoder 300 when decoding video data.

[0066] To perform CABAC, video encoder 200 may assign a context within a context model to a character to be transmitted. The context may relate, for example, to whether adjacent symbol values are null or not. The probability determination may be based on the context assigned to the symbol.

[0067] The video encoder 200 may further generate syntax data, such as block-based syntax data, picture-based syntax data, and sequence-based syntax data, for the video decoder 300, for example, in a picture header, a block header, a slice header, or other syntax data such as sequence parameter set (SPS), picture parameter set (PPS), or video parameter set (VPS). The video decoder 300 may also decode such syntax data to determine how to decode the corresponding video data.

[0068] Thus, video encoder 200 can generate a bitstream including encoded video data, such as syntax elements describing the partitioning of a picture into blocks (eg, CUs), and prediction information and/or residual information for the blocks. Ultimately, the video decoder 300 may receive the bitstream and decode the encoded video data.

[0069] In principle, the video decoder 300 performs a reverse process to the process performed by the video encoder 200 to decode encoded video bitstream data. For example, video decoder 300 may decode values for bitstream syntax elements using CABAC in a substantially similar, albeit inverse manner to the CABAC coding process of video encoder 200. The syntax elements may define splitting information of a picture into CTUs and splitting each CTU according to an appropriate splitting structure, such as a QTBT structure, to define a CU in a CTU. The syntax elements may further define prediction and residual information for blocks (eg, CUs) of video data.

[0070] The residual information may be represented, for example, by quantized transform coefficients. Video decoder 300 may inversely quantize and inversely transform the quantized block transform coefficients to reconstruct a residual block for that block. Video decoder 300 uses the signaled prediction mode (intra or inter prediction) and associated prediction information (eg, motion information for inter prediction) to generate a prediction block for that block. Video decoder 300 may then combine the prediction block and the residual block (sample-to-sample basis) to reconstruct the original block. Video decoder 300 may perform additional processing, such as performing a deblocking process to reduce visual artifacts along block boundaries.

[0071] The present disclosure may generally refer to "signaling" certain information, such as syntax elements. The term “signaling” may generally refer to reporting values for syntax elements and/or other data used to decode encoded video data. That is, video encoder 200 may signal values for syntax elements in the bitstream. In general, signaling refers to the generation of a value in a bitstream. As noted above, source device 102 may transport the bitstream to destination device 116 in substantially real time or non-real time, as may occur when syntax elements are stored in storage device 112 for later retrieval by destination device 116.

[0072] FIG. 2A and 2B are a conceptual diagram illustrating an exemplary quadtree-binary tree (QTBT) structure 130 and corresponding coding tree unit (CTU) 132. Solid lines represent quadtree partitioning, and dotted lines indicate binary tree partitioning. At each split (ie, non-leaf) node of the binary tree, one flag is signaled indicating which type of split (ie, horizontal or vertical) is used, where 1 indicates horizontal split and 0 indicates vertical split in this example. To split a quadtree, it is not necessary to specify a split type, since the quadtree nodes divide the block horizontally and vertically into 4 equally sized subblocks. Accordingly, the video encoder 200 may encode and the video decoder 300 may decode syntax elements (such as split information) for the tree layer of the area of the QTBT 130 structure (i.e., solid lines) and syntax elements (such as split information) for the prediction tree layer. structures QTBT 130 (ie dashed lines). Video encoder 200 may encode and video decoder 300 may decode video data, such as prediction and transform data, for the CUs represented by leaf leaf nodes of the QTBT structure 130.

[0073] In general, CTU 132 in FIG. 2B may be associated with parameters defining block sizes corresponding to the nodes of the QTBT 130 structure at the first and second levels. These parameters may include CTU size (representing the size of CTU 132 in samples), minimum quadtree size (MinQTSize, representing the minimum allowable quadtree leaf node size), maximum binary tree size (MaxBTSize, representing the maximum allowed binary tree root node size), maximum the depth of the binary tree (MaxBTDepth, representing the maximum allowable depth of the binary tree); and the minimum size of the binary tree (MinBTSize, representing the minimum allowable size of the leaf node of the binary tree).

[0074] The root node of the QTBT structure corresponding to the CTU may have four child nodes at the first level of the QTBT structure, each of which may be partitioned according to a quadtree split. That is, the first level nodes are either leaf nodes (having no children) or have four children. An example structure of QTBT 130 represents nodes such as including a parent node and child nodes having solid lines for branches. If the nodes of the first level are not larger than the maximum allowable size of the root node of the binary tree (MaxBTSize), then the nodes can be further split into the corresponding binary trees. Splitting a binary tree of one node may be repeated until the nodes resulting from the split reach the minimum allowable binary tree leaf node size (MinBTSize) or the maximum allowable binary tree depth (MaxBTDepth). An example structure of QTBT 130 represents such nodes as having dashed lines for branches. A leaf node of a binary tree is referred to as a coding unit (CU) which is used for prediction (eg intra-picture or inter-picture prediction) and transformation without any further partitioning. As discussed above, CUs may also be referred to as “video blocks” or “blocks”.

[0075] In one example of the QTBT partitioning structure, the CTU size is set to 128x128 (luminance samples and two corresponding 64x64 chrominance samples), MinQTSize is set to 16x16, MaxBTSize is set to 64x64, MinBTSize (as for width, and height) is set to 4, and MaxBTDepth is set to 4. Quadtree partitioning is applied to the CTU first to form the leaf nodes of the quadtree. The leaf nodes of the quadtree can be from 16x16 (ie MinQTSize) to 128x128 (ie CTU size). If a quadtree leaf node is 128x128, it will not be further split into a binary tree because the size is larger than MaxBTSize (ie 64x64, in this example). Otherwise, the leaf node of the quadtree will be further split into a binary tree. Thus, the leaf node of the quadtree is also the root node for the binary tree and has a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (4, in this example), no further splitting is allowed. When a binary tree node has a width equal to MinBTSize (4, in this example), this means that no further vertical splitting is allowed. Similarly, a binary tree node having a height equal to MinBTSize implies that no further horizontal splitting is allowed for that binary tree node. As noted above, the leaf nodes of the binary tree are referred to as CUs and are further processed according to prediction and transformation without further partitioning.

[0076] Blocking artifacts may include horizontal and vertical discontinuities (discontinuities) in the reconstructed picture that do not exist in the original still or video picture and are often the result of moderate to high compression. For example, if the video encoder highly compresses the input image, then the visual quality of the decoded image suffers, so that when the image content is decompressed (decompressed), i.e., decoded, the content of the reconstructed image suffers from blocking artifacts. These artifacts in flat areas look like "mosaic" because the artifacts are not masked by high contrast content. In addition, blocking artifacts in video playback can be observed as “moving and flickering” as the irregularities are located differently in successive frames.

[0077] As discussed above, one source of blocking artifacts is block-based transform coding, including transform and quantization, that is performed on intra- and inter-prediction errors (eg, residuals). Coarse quantization of the transform coefficients can cause a visual distribution of inhomogeneities at the block boundaries. Motion compensated prediction is another potential source of blocking artifacts. Motion compensated blocks are generated by copying interpolated pixel data from different locations, possibly different reference frames. Since there is almost never a perfect match for this data (eg, sample values for reference blocks derived from interpolation), discontinuities usually occur at the boundaries of the copied data blocks. That is, two adjacent blocks encoded using motion compensation may exhibit discontinuities.

[0078] Deblocking filtering (eg, as in HEVC) is typically performed after a picture or group of blocks is reconstructed/decoded to reduce blocking artifacts. More specifically, deblocking filtering changes the values of the samples located near each block boundary, by filtering, clipping, or other means, so that the inhomogeneities are smoothed out and thus become less visible.

[0079] In video compression, deblocking filtering can be performed on images in the image buffer and outside the prediction loop, which means that deblocked filtered pictures are not used to predict other pictures. Deblocking filtering can also be performed in a prediction loop, which means that deblocking pictures are used as reference pictures to compensate for the motion of future pictures. Both techniques improve the subjective quality of the displayed video, although the latter potentially also improves compression performance as a result of the inter-prediction accuracy improved by the use of deblocking reference frames.

[0080] Deblocking in HEVC is an in-loop process and is applied to block boundaries that satisfy two conditions. The first condition is that the boundaries are CU, PU, or TU boundaries, and the second condition is that the x-coordinate (or y-coordinate) is (and in some cases must be) a multiple of 8 if the boundary is vertical ( or horizontal). The second condition means that the minimum distance of two parallel adjacent borders to be deblocked is 8 pixels, which provides better parallelization, as described in more detail below. The deblocking filter divides the border to be processed, regardless of the length of the border, into a plurality of non-overlapping segments of 4 samples, which are the units on which the deblocking filter performs deblocking filtering. In the present disclosure, deblocking operations on a segment will be introduced, and a segment may be assumed to be vertical, however, the processing of horizontal segments is essentially the same.

[0081] The video decoder 300, when performing deblocking filtering, may perform edge strength (strength) determination. For a segment to be processed, the video decoder examines the encoding conditions (eg, motion vector (MV), reference index, and presence of non-zero transform coefficients) of two blocks on each side, denoted P and Q, on each side of the block boundary.

[0082] FIG. 3 shows an example of a P box 142 and a Q box 144. P box 142 is an 8×8 block and Q box 144 is a 16×16 block. P-box 142 and Q-box 144 share a border 146, which is shown in FIG. 3 as segments 146A and 146B. Segments 146A and 146B are both four samples long, in accordance with HEVC deblocking filtering, but the techniques of the present disclosure are not limited to a particular segment length.

[0083] For luma blocks, the video decoder determines the strength of the segment boundary 146A. The edge strength represents how intense blocking artifacts should appear around segment 146A. In one example, the edge strength value may be 0, which means that the coding conditions in P-box 142 and Q-box 144 are such that no blocking artifacts are expected along segment 146A, and deblocking filtering for segment 146A may be omitted. In another example, the edge strength value may be 2, which means that the coding conditions in P-box 142 and Q-box 144 are such that strong deblocking artifacts are potentially expected, and stronger deblocking filtering may be desired. The border strength value may also be 1, which means that the coding conditions in P-box 142 and Q-box 144 are such that less severe blocking artifacts are potentially expected, and some deblocking filtering may be desirable, but not as strong as deblocking filtering when edge strength is 2. Details of edge strength derivation are described in section 8.7.2.4 of HEVC.

[0084] The video decoder may determine edge strengths for chrominance blocks differently than for luma blocks. For example, for chrominance blocks, the video decoder assigns segments adjacent to intra-coded blocks a border sig equal to 2, and otherwise assigns a border strength equal to 0 to the segments.

[0085] As described above, the video decoder may estimate the likelihood that a segment has blocking artifacts based on a coding condition of adjacent blocks. However, the video decoder may also perform further analysis, based on sample values near such a segment, to determine whether or not to deblock the segment. First, the video decoder decides whether to filter a segment by calculating the second derivatives of the four samples near the segment.

[0086] FIG. 4 shows an example of a segment 150 that is on the boundary between P-box 152 and Q-box 154. The video decoder calculates the second derivatives of the four samples near the segment, which are shown in FIG. 4 as circled samples 158A-158D. If the sum of the four second derivatives (see Eq. (1)) is less than the threshold β, as described below with respect to the thresholds β and t _C , then the video decoder determines that the segment requires deblocking. Otherwise, the video decoder determines that the segment is considered to be located in a non-planar region where blocking artifacts are likely to be masked and therefore do not require deblocking. For a segment, even with a non-zero edge strength, the video decoder may skip deblocking filtering if the threshold according to Eq. (1) not reached.

[0087] Second, for the segment to be deblocked, the video decoder may make a different decision whether to use heavy or normal filtering mode. If the following six conditions (Eq. (2-1)-(2-6)) are all true, which means that the area is probably too smooth to mask any blocking artifacts, then the video decoder determines that the heavy mode is used. filtration. Otherwise, the video decoder determines that the normal filtering mode is used.

The threshold parameter t _C represents a cutoff parameter, which is described in more detail below.

[0088] FIG. 5 shows an example of a segment 160 that is on the border between P-box 162 and Q-box 164. FIG. 5 will be used to show the deblocking filtering operation for a single row represented by the physical sample positions p ₀ , p ₁ , p ₂ , p ₃ , q ₀ , q ₁ , q _{2 ,} and q ₃ .

[0089] For the edge strength determinations and other brightness decisions described above, the video decoder performs analysis and inference at the segment level. In the intensive filtering mode for luminance and the normal filtering mode for luminance, deblocking filtering is performed line by line (for example, row by row if the segment is vertical, or column by column if the segment is horizontal).

[0090] In the intensive filtering mode for luminance, the video decoder processes three samples on each side of segment 160. In the example of FIG. 5, the sample values p ₀ , p ₁ and p ₂ in P-block 162 are updated to p ₀ ^' , p ₁ ^' and p ₂ ^' by low-pass filtering as shown in Eq. (3-1)-(3-3).

Changed sample values p _i ^' ( i =0, 1, 2) are limited (cut off) to the range [ p _i - 2 t _C , p _i +2 t _C ].

[0091] The video decoder processes q ₀ , q ₁ and q ₂ in Q-block 164 using essentially the same equations as Eq. (3-1)-(3-3), albeit for q instead of p . Clipping is applied in the same way for q as it is for p .

[0092] In the normal filtering mode for luma, the video decoder processes one or two samples on each side of segment 160. For the left side of segment 160, for example, the video decoder checks the conditions in Eq. (4-1). If the conditions are met, then the video decoder processes the samples p ₀ and p ₁ . Otherwise, the video decoder only processes the sample p ₀ . Similarly, for the right side, the video decoder checks the conditions in Ur. (4-2) to determine if sample q ₁ is being processed in addition to sample q ₀ . Since the number of samples to be processed is decided independently on each side of the segment, it is possible to process one sample on one side and two samples on the other side.

To process p ₀ and q ₀ , an intermediate value δ is first calculated, as in Eq. (5).

[0093] If the absolute value of δ is greater than or equal to 10 times t _C , then the boundary is considered as a natural edge to be preserved, and the video decoder does not perform deblocking filtering on the current line. Otherwise, the video decoder limits δ in the range -t _C to t _C , as shown in Eq. (6).

[0094] The video decoder updates the values of p ₀ and q ₀ to p ₀ ^' and q ₀ ^' , respectively, by adding and subtracting Δ ₀ as shown in Eq. (7-1) and (7-2).

[0095] To process the second sample on each side of segment 164, i.e., p ₁ and q ₁ , the video decoder uses Eq. (8-1) and (8-2) to get the updated p ₁ ^' and q ₁ ^' values.

[0096] The video decoder performs chrominance deblocking filtering by determining the edge strength value, but without performing any analysis of the sample values. The video decoder processes only the first sample on each side of the segment, i.e., p ₀ and q ₀ , using Eq. (7-1) and (7-2), where delta Δ ₀ is calculated as in Eq. (9).

[0097] To avoid excessive filtering, the video decoder calculates two parameters β and t _C . The video decoder uses a threshold β to control the way deblocking is performed, such as whether to deblock a segment, whether to use heavy or normal filtering mode, and/or whether to process one or two samples on one side of a segment. If β or scaled β is reached or exceeded (see Eq. (1), (2) and (4)), which means more change in local sample values, then deblocking tends to be more conservative to preserve detail in the original image. Otherwise, the local sample values change less (ie, smoother) and the video decoder performs deblocking filtering in a more aggressive manner.

[0098] The video decoder uses the cutoff value t_C, mainly to control the maximum change in sample size, with the exception of Eq. (2-5) and (2-6). In HEVC, for example, in normal or chrominance filtering mode, the change in sample size is limited such that the change does not exceed ±t_C for the first sample on one side of the segment (applicable to luma and chrominance), or ±t_C/2 for the second sample (only applies to brightness). In HEVC, for heavy filtration mode, where a larger change in magnitude is implied, the maximum change is limited to ±2t_C for three samples processed on each side of the segment.

[0099] The values of β and t _C mainly depend on the values of the quantization parameter (QP) from the left P-box and the right Q-box. More specifically, the video decoder may use the average QP of P and Q, denoted as QP _ave = ( QP _P + QP _Q +1)>>1, as an index to look up two 1-D lookup tables (LUTs) for β and t _C , respectively. Although the search index for finding the value of t _C can be tuned by adding two, i.e., ( QP _ave +2) if the boundary strength is 2, the predominant factor determining the values of β and t _C is still QP _ave . In both LUTs, the element values increase monotonically as the search indexes increase, which means that the higher the QP _ave , the larger the values of β and t _C will be. Thus, more intense deblocking filtering is more likely to be selected, and a larger change in magnitude is allowed. Lower QP, on the contrary, leads to smaller or even zero values for β and t _C . When encoded at low QP, the picture generally has less blocking artifacts or less pronounced blocking artifacts and therefore requires less or no deblocking.

[0100] The indexes used to search for β and t _C in the LUT, denoted as idx β and idx _tc , can be further configured by two parameters tc_offset_div2 and beta_offset_div2, respectively (see equations (10-1) and (10-2)),

where tc_offset_div2 and beta_offset_div2 are sent in the slice header or picture parameter set (PPS). This allows the encoder to adapt the deblocking rate depending on the characteristics of the sequence, the coding mode, and other factors.

[0101] Deblocking according to HEVC has two consecutive stages, which can enable parallelization. In the first stage, the video decoder filters all vertical block boundaries, and in the second stage, the video decoder filters all horizontal block boundaries. In the second stage, the samples used to make the mode and filter decisions are the outputs of the first stage. At each stage where the deblocking boundaries are all parallel and at least 8 samples apart, the deblocking samples of one boundary do not overlap with the deblocking samples of any other boundaries. In this context, the samples involved in single boundary deblocking include up to three samples to be filtered on each side of the boundary and up to four samples on each side to support filtering and mode decision making, and therefore single boundary deblocking can run in parallel with any other boundaries.

[0102] The HEVC deblocking filter filters samples near a block boundary and clips changes in sample values that are greater than the threshold values. The HEVC deblocking filter operates in three modes with three different filter intensity levels. The three modes, or filter intensities, are referred to as strong, normal, and zero (i.e., no filtering), with increasing local sampling activities near the block boundary (see equations (1) and (2-1)-(2-6 )). In the strong filtering mode, the HEVC deblocking filter filters three samples on each side of the block boundary by low-pass filtering (see Equations (3-1)-(3-3)). For normal filtering mode, the HEVC deblocking filter filters at least the sample closest to the boundary, and on each side of the boundary may also filter the second closest sample if the inner samples are smooth enough (see Equations (4-1) and (4- 2)). The HEVC deblocking filter performs a clipping controlled by the parameter t _C . For strong filtering, the change in sample size is limited to not exceed 2t _C . For normal filtering, the magnitude changes of the first and second samples are limited to not exceed t _C and t _C /2, respectively (see Eq. (6)-(8-2)). Different border strength values create a difference in t _C , i.e., a segment with a border strength of 2 has a larger t _C than a segment with a border strength of 1.

[0103] The HEVC deblocking filter essentially follows the general H.264/AVC deblocking scheme and inherits the main features of the H.264/AVC deblocking filter, such as edge strength determinations based on coding conditions, multi-level filtering strengths from strong to zero, QP and the strength of the boundary, depending on the parameters β and t _C . Compared with the H.264/AVC deblocking filter, the newly developed elements in the HEVC deblocking filter allow easier parallel processing and are better adapted to the HEVC jumbo block coding structure, but do not greatly improve coding efficiency. Therefore, HEVC deblocking, which has been seen as a good compromise between computational complexity and coding efficiency during HEVC finalization, may be oversimplified given the capabilities of today's highly advanced hardware. The techniques of the present disclosure potentially benefit from large hardware computing resources to achieve significant improvements in coding efficiency while maintaining a parallelization-friendly structure.

[0104] The present disclosure provides techniques that can improve aspects of deblocking filtering, including HEVC deblocking filtering. One example of a disadvantage of HEVC deblocking filtering is that using only three levels to represent the smoothness of the boundary region corresponding to three levels of filtering intensity can be too coarse. Another example of a disadvantage of HEVC deblocking filtering is that only two of the four segment rows are used in the deblocking mode decision and filter selection, and only four samples per side are used per row. Since the next generation video codec block size can be up to 128×128 or even larger, using such a small fraction of samples for mode decision may not accurately reflect the actual activity of the boundary region and may be noise sensitive.

[0105] Another example of the disadvantage of HEVC deblocking filtering is that the samples to be filtered on each side of the segment may include 3 samples or one sample, which may not be enough to provide good results for large block deblocking. Another example of the disadvantage of HEVC deblocking filtering is that there are, in general, five predefined 4-tap or 5-tap filters pre-assigned to three samples in the heavy filtering mode (see Equations (3-1)-(3- 3)) and two samples in the normal filtering mode (see equations (5)-(8-2)), respectively. The limited number of options, short length, and inflexibility in the selection of the HEVC deblocking filter can result in poor deblocking performance.

[0106] Another example of a disadvantage of HEVC deblocking filtering is that segments with different edge strengths can have quite different local activities but share the same deblocking filters, although the cutoff value is larger for a border strength of 2. Another An example of a disadvantage of HEVC deblocking filtering is that the HEVC deblocking filter does not differentiate between a P-box and a Q-box. The filtration intensity is determined by the average of the second derivative of P and Q (see equations (1) and (2-1)-(2-2)), and the values of the parameters β and t _C depend on the average QP P and Q. Losses in efficiency may occur. in P-box and Q-box processing is the same for coding scenarios, where one of the P-box or Q-box can be smooth and the other is rich in detail, which is not an uncommon scenario.

[0107] The present disclosure proposes various techniques for potentially improving the performance of deblocking filtering used for video coding or video processing. According to one example, video decoder 300 may be configured to perform filter analysis (eg, filter selection or filter intensity adaptation) based on block splitting parameters. In one example, video decoder 300 may be configured to determine a deblocking filter length and filter support (eg, downsampling filter support) based on a block length orthogonal to a filtered boundary. In another example, the length of the block aligned with the filtered boundary influences the decision process, such as allowing spatial sub-sampling of the boundary samples used to make the filter decision.

[0108] According to another example, the video decoder 300 may be configured to adapt the output sample confidence interval, also known as the Tc coefficient, as a function of the block partition parameters (e.g., sizes) and associated position within the block of the current sample being processed, to which the limitation parameter Tc is applied. Tc limits the deviation, positive or negative, of the deblocking sample value from the decoded sample, and is input to the deblocking process.

[0109] According to another example, the video decoder 300 may be configured to use processing block parameters (block size) to limit the set of samples available to the decision and filter process while maintaining the structure of the decision and filter process. The video decoder 300 may generate samples that are required for the filtering process but are not available due to limitations using a given process, such as an extrapolation or padding process. According to another example, video decoder 300 may use transform properties such as transform type/basic function information and transform coefficients signaled as non-zero to video decoder 300 and therefore present in the reconstructed block of samples to select deblocking filter parameters. , such as filter type, filter length, Tc clipping parameters, skip space spacing. According to another example, video decoder 300 may use the P- and Q-block quantization parameters to set the P and Q filter selection thresholds independently.

[0110] In one example, for a block boundary separating blocks of different sizes, video decoder 300 may decide on asymmetric deblocking filtering. For example, when a block boundary separates blocks of different sizes, video decoder 300 may select an asymmetric deblocking filter.

[0111] In another example, if the video decoder 300 determines that an asymmetric filter should be used, then the deblocking process on both sides of the boundary may be different. For example, deblocking filter parameters such as tap length, filter factor, clipping or normalization process may be different on both sides of the filtered boundary. Additionally or alternatively, parameters for decision processes such as analysis filter tap lengths, analysis filter coefficients, cutoff, normalization, or thresholds may be different on both sides of the filtered boundary.

[0112] In another example, video decoder 300 may determine parameters for asymmetric filtering based on one or more block coding mode parameters at the boundary. Examples of such coding mode parameters include coding mode, prediction mode, slice type, or other such parameters.

[0113] Non-limiting examples of implementations of the techniques introduced above will now be described. In one example, the filter analysis process (filter selection or filter intensity adaptation) is extended to include block splitting parameters. For example, video decoder 300 may be configured to determine a deblocking filter length and filter support (eg, downsampling filter support) based on a block length orthogonal to a filtered boundary. In another example, video decoder 300 may be configured to make a decision based on a block length aligned with a filtered boundary such that spatial sub-sampling of the boundary samples used to make a filter decision is allowed.

[0114] The following pseudo-code provides an exemplary implementation of the deblocking parameter derivation performed by the video decoder 300:

VERT_SAMPLE_SKIP is a variable defining the spatial resampling parameters (number of skipped samples) that specifies the number of samples used to classify edge samples.

DB_BLOCK_SIZE_CLASSIFIER is a variable that specifies the number of samples used to classify edge samples.

The function is SmoothAsymArea(const CodingUnit& cu, const DeblockEdgeDir edgeDir, const Position& localPos)

[0115] In the examples above, pSrcP may, for example, represent a pointer to a block on one side of the boundary to be deblocked, and pSrcQ may represent a pointer to a block on the other side of the boundary to be deblocked.

[0116] The video decoder 300 may be configured to determine an output sample confidence interval, also known as a coefficient Tc, to limit the deviation of the deblocked sample value from the decoded sample. Tc may be introduced into the deblocking process. Video decoder 300 may be configured to adapt Tc as a function of block splitting parameters, such as block size, and an associated position within a block or within a filtered boundary of the current processing sample to which the Tc constraint parameter is applied. In one example, the associated position of the current sample being processed is the location of the current sample being processed relative to the filtered boundary.

[0117] In one exemplary implementation, the constraint parameter Tc can be defined as a function of the associated position of the current filtered row/column within the filtered boundary, where i is the index of the currently processed row/column of the filtered boundary, starting from the top/right boundary location, respectively.

[0118] The following pseudo-code provides an exemplary implementation of the application of Tc:

In some examples, the value of Tc may be determined for a block as a function of the position associated with the top-left position of the currently processed block:

y=coordinateCurrentlyProcessedSample.y -

coordinateTopLeftSampleOfBlock.y;

x=coordinateCurrentlyProcessedSample.x -

coordinateTopLeftSampleOfBlock.x;

delta1=Clip3(-tc2[y][x], tc2[y][x], delta[y][x]);

[0119] The video decoder 300 may be configured to use processing block parameters, such as block size, to limit the set of samples available to the decision and filter process while maintaining the structure of the decision and filter process. The video decoder 300 may generate samples that are required for the filtering process but are not available due to limitations using a given process, such as an extrapolation or padding process.

[0120] The following pseudo-code provides an exemplary implementation. In some implementations, a filter group may be defined to handle the border. The choice of the filter may be based on the union of the block sizes adjacent at the block boundary and the classification result of the boundary samples.

DB_FILTER_BLOCK_SIZE1, DB_FILTER_BLOCK_SIZE2 and DB_FILTER_BLOCK_SIZE3 are variables, each of which defines the number of samples used to filter edge samples, defining filter support, for example 3 filters.

[0121] Video decoder 300 may be configured to use transform properties, such as transform type/basis function information, and transform coefficients signaled as non-zero to video decoder 300 and therefore present in the reconstructed block of samples, to select elimination filter parameters. blockiness, such as filter type, filter length, Tc constraint parameters, skip spacing.

[0122] The following pseudo-code provides an exemplary implementation. Assume that video encoder 200 and video decoder 300 use a set of multiple transforms, such as {T1,T2,T3}, where video encoder 200 selects the optimal transform applicable to the horizontal and vertical directions. In the video decoder 300, the applicability of the transform to the horizontal/vertical direction can be inferred from the syntax elements of the bitstream. The inference process used in WD VVC (Draft 2) is specified below.

Table 1 Conversion and signaling display table MTS_CU
_flag MTS_Hor
_flag MTS_Ver
_flag Intra Inter Horizon-
tal verti-
caloric Horizon-
tal verti-
caloric 0 DCT2 1 0 0 DST7 DST7 DCT8 DCT8 0 1 DCT8 DST7 DST7 DCT8 1 0 DST7 DCT8 DCT8 DST7 1 1 DCT8 DCT8 DST7 DST7

[0123] The following pseudo-code can be used to derive deblocking filter parameters:

}

[0124] The following is an example of the ApplyDeblockingProcessType1 function. The structure of other functions, such as deblocking functions, may also follow a similar pattern.

ApplyDeblockingProcessType1()

{

1. Derive the classifier length based on the block size orthogonal to the block boundary, the transform type, and the index of non-zero encoded transform coefficients and local activity score.

2. Derive the deblocking length based on the block size orthogonal to the deblocking boundary, the transform type, and the index of non-zero encoded transform coefficients and local activity score.

3. Derive the samples skipped during the classifier or deblocking process based on the block size orthogonal to the deblocking boundary, the transform type, and the index of non-zero encoded transform coefficients and local activity score.

4. Derive deblocking parameters such as Tc, local thresholds, QP tuning based on block size orthogonal to deblocking boundary, transform type and index of non-zero encoded transform coefficients, and local activity estimate.

[0125] The video decoder 300 may be configured to use the P- and Q-block quantization parameters to set the P- and Q-filter selection thresholds independently.

iQPp=cuP.qp;

iQPq=cuQ.qp;

[0126] The following pseudo-code is defined for the P-box parameters. The processing for the Q-block may be similar.

[0127] In yet another example, the following implementation of position-dependent deblocking can be used in conjunction with other features disclosed here.

[0128] Let numberQSide and numberPSide be the lengths of the filters applied to the current row. As introduced above, the numberPSide and numberQSide filter lengths can be based on the block size. In this example, two filters are used to filter 7 and 3 pixels of the box border. Position-dependent clipping value parameters are expressed in terms of tables Tc7 and Tc3.

[0129] In the above example, "Src=piSrcP[-iOffSet*thePos]" illustrates the value of the sample before deblocking filtering. “((refMiddle*dbCoeffsP[thePos] + refP * (64 - dbCoeffsP[thePos]) + 32) >> 6)” illustrates the application of the deblocking filtering operation. “refP” or “refQ” represents the value of the linear unions of the samples on one side (P- or Q-box side) of the box boundary, or the filtered value on one side (P- or Q-box side) of the box boundary. "refMiddle" represents the value of the linear unions of the samples on both sides of the block boundary, or the filtered value on both sides of the block boundary. “Clip3(src - cvalue, src+cvalue, ((refMiddle*dbCoeffsP[thePos] + refP * (64 - dbCoeffsP[thePos]) + 32) >> 6))” illustrates clipping the deblocking filtered value in the range src- value and src+cvalue which is the sample value before filtering plus/minus the cvalue threshold, where the cvalue is adapted based on the sample location and block size.

[0130] As discussed above, video decoder 300 may determine numberQSide and numberPSide based on the block size to which deblocking filtering is applied. refMiddle represents the value after applying the filter to the samples on each side of the P- and Q-box boundary. refP and refQ represent the values after filters have been applied to the samples in the P-box and Q-box.

[0131] In yet another example, the following implementation of position-dependent deblocking can be used in conjunction with the other features described here.

[0132] Let numberQSide and numberPSide be the length of the filter applied to the current row. This example uses two filters applied to filter 7 and 3 pixels of the box border. Position-dependent clipping value parameters are expressed by tables Tc7 and Tc3.

[0133] In the above example, "Src=piSrcP[-iOffSet*thePos]" illustrates the value of the sample before deblocking filtering. “((refMiddle*dbCoeffsP[thePos] + refP * (64 - dbCoeffsP[thePos]) + 32) >> 6)” illustrates the application of the deblocking filtering operation. “refP” and “refMiddle” are examples of linear unions of samples on one side of a box boundary, or filtered pixels on one side of a box boundary. “Clip3(src - cvalue, src+cvalue, ((refMiddle*dbCoeffsP[thePos] + refP * (64 - dbCoeffsP[thePos]) + 32) >> 6))” illustrates clipping the deblocked filtered value in the range src- value and src+cvalue which is the sample value before filtering plus a threshold cvalue where the cvalue is adapted based on the sample location and block size.

[0134] As discussed above, video decoder 300 may determine numberQSide and numberPSide based on the block size to which deblocking filtering is applied. refMiddle represents the value after applying the filter to the sample on each side of the P-box and Q-box boundary. refP and refQ represent the values after filters have been applied to the samples in the P-box and Q-box.

[0135] In another example, the following implementation of position-dependent deblocking can be used in conjunction with other features described here.

[0136] Let numberQSide and numberPSide be the lengths of the filters applied to the current row. This example uses two filters applied to filter out the 7 and 3 pixel borders of the box. The position dependent cutoff parameters are expressed in terms of tables Tc7 and Tc3.

[0137] In yet another example, the following asymmetric filtering definition condition may be used in conjunction with other features disclosed herein.

Disable long filter in asymmetric case depending on P and Q prediction modes.

[0138] FIG. 6 is a block diagram illustrating an exemplary video encoder 200 that may perform the techniques of the present disclosure. Fig. 6 is provided for purposes of explanation and should not be construed as limiting the techniques broadly illustrated and described in this disclosure. For purposes of explanation, the present disclosure describes the video encoder 200 in the context of video coding standards such as the HEVC video coding standard and the upcoming H.266 video coding standard. However, the techniques of the present disclosure are not limited to these video coding standards and apply in general to video coding and decoding.

[0139] In the example of FIG. 6, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216. , a decoded picture buffer (DPB) 218 , and an entropy encoding module 220 .

[0140] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. Video encoder 200 may receive video data stored in video data memory 230 from, for example, video source 104 (FIG. 1). DPB 218 may act as a reference picture memory that stores reference video data for use in prediction of subsequent video data by video encoder 200. Video data memory 230 and DPB 218 may be formed by any of a variety of memory devices such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. . Video data memory 230 and DPB 218 may be provided by the same memory device or by separate memory devices. In various examples, video data memory 230 may be on-chip with other components of video encoder 200, as illustrated, or off-chip with respect to those components.

[0141] In this disclosure, reference to video data memory 230 is not to be interpreted as being limited to memory internal to video encoder 200, unless specifically described as such, or memory external to video encoder 200, unless specifically described as such. . Rather, reference to video data memory 230 is to be understood as a reference memory that stores video data that video encoder 200 receives for encoding (eg, video data for the current block to be encoded). Memory 106 in FIG. 1 may also provide temporary storage of output data from various modules of the video encoder 200.

[0142] The various modules in FIG. 6 are illustrated to aid in understanding the operations performed by the video encoder 200. Modules can be implemented as fixed function circuits, programmable circuits, or a combination of both. Fixed function charts refer to circuits that provide certain functionality and are preconfigured for operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, the programmable circuits may execute software or firmware that causes the programmable circuits to operate in the manner specified by the instructions of the software or firmware. Fixed function circuits can execute program instructions (for example, accept parameters or output parameters), but the types of operations that fixed function circuits perform are usually immutable. In some examples, one or more modules may be separate circuit blocks (fixed functional blocks or programmable blocks), and in some examples, one or more modules may be integrated circuits.

[0143] Video encoder 200 may include arithmetic logic units (ALUs), elementary function units (EFUs), digital circuits, analog circuits, and/or programmable cores formed by programmable circuits. In examples where operations of video encoder 200 are performed using software executable by programmable circuitry, memory 106 (FIG. 1) may store instructions (eg, object code) of the software that video encoder 200 receives and executes, or other memory in the encoder. 200 videos (not shown) can store such instructions.

[0144] The video data memory 230 is adapted to store received video data. The video encoder 200 may extract a picture of the video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202 . The video data in the video data memory 230 may be raw video data to be encoded.

[0145] The mode selection module 202 includes a motion estimation module 222, a motion compensation module 224, and an intra-prediction module 226. The mode selection module 202 may include additional functional modules for performing video prediction in accordance with other prediction modes. By way of example, mode select module 202 may include a palette module, an intrablock copy module (which may be part of motion estimation module 222 and/or motion compensation module 224), an affine module, a linear model (LM) module, or the like.

[0146] Mode selection module 202 typically coordinates multiple encoding passes to test combinations of encoding parameters and the resulting rate-distortion values for such combinations. The coding parameters may include splitting the CTU into CUs, prediction modes for the CUs, transform types for the CU residual data, quantization parameters for the CU residual data, and so on. The mode selector 202 may ultimately select a coding parameter aggregation having rate-distortion values that are better than the other aggregations being tested.

[0147] The video encoder 200 may split the picture retrieved from the video data memory 230 into a sequence of CTUs and encapsulate one or more CTUs within the slice. The mode selector 202 may partition the CTUs of the pictures according to a tree structure such as the QTBT structure or the HEVC quadtree structure described above. As described above, video encoder 200 may generate one or more CUs from a split of CTUs according to a tree structure. Such a CU may also be referred to generically as a “video block” or “block”.

[0148] In general, mode selection module 202 also controls its components (e.g., motion estimation module 222, motion compensation module 224, and intra-prediction module 226) to generate a prediction block for the current block (e.g., current CU or, in HEVC , overlapping part of PU and TU). For inter-prediction of the current block, motion estimation module 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (eg, one or more of the previously encoded pictures stored in DPB 218). In particular, the motion estimation module 222 may calculate a value representing how similar the potential reference block is to the current block, for example, according to the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), rms differences (MSD) or the like. Motion estimation module 222 can typically perform these calculations using the sample-to-sample differences between the current block and the reference block under consideration. Motion estimation module 222 may identify the reference block having the smallest value obtained from these calculations, indicating the reference block that most closely matches the current block.

[0149] The motion estimation module 222 may generate one or more motion vectors (MVs) that define the positions of the reference blocks in the reference pictures relative to the position of the current block in the current picture. Motion estimation module 222 may then provide motion vectors to motion compensation module 224 . For example, for unidirectional inter prediction, motion estimation module 222 may provide one motion vector, while for bidirectional inter prediction, motion estimation module 222 may provide two motion vectors. Motion compensation module 224 may then generate a prediction block using the motion vectors. For example, motion compensation module 224 may extract reference block data using a motion vector. As another example, if the motion vector has fractional sampling precision, motion compensation module 224 may interpolate values for the prediction block in accordance with one or more interpolation filters. In addition, for bi-directional inter-prediction, motion compensation module 224 may extract data for two reference blocks identified by respective motion vectors and combine the extracted data, for example, by sample-to-sample averaging or weighted averaging.

[0150] As another example, for intra-prediction or intra-prediction coding, intra-prediction module 226 may generate a prediction block from samples adjacent to the current block. For example, for directional modes, intra-prediction module 226 can typically mathematically combine adjacent sample values and populate the current block with these computed values in a particular direction to form a prediction block. As another example, for the DC mode, intra-prediction module 226 may compute an average of adjacent samples for the current block and generate a prediction block to include that average for each prediction block sample.

[0151] The mode selection module 202 provides a prediction block to the residual generation module 204 . Residual generation module 204 receives a raw, uncoded version of the current block from video data memory 230 and a prediction block from mode selection module 202 . The residual generation module 204 calculates the sample-to-sample difference between the current block and the prediction block. The resulting sample-to-sample differences determine the residual block for the current block. In some examples, residual generation module 204 may also determine differences between sample values in the residual block to generate a residual block using residual differential pulse code modulation (RDPCM). In some examples, the remainder generator 204 may be formed using one or more subtraction circuits that perform binary subtraction.

[0152] In examples where mode selector 202 partitions a CU into PUs, each PU may be associated with a luminance prediction unit and corresponding chrominance prediction units. Video encoder 200 and video decoder 300 may support PUs having different sizes. As mentioned above, the CU size may refer to the size of the luminance coding unit of the CU, and the size of the PU may refer to the size of the luminance prediction unit of the PU. Assuming a particular CU size is 2Nx2N, video encoder 200 may support 2Nx2N or NxN PU sizes for intra prediction and symmetrical PU sizes of 2Nx2N, 2NxN, Nx2N, NxN or similar. for inter-prediction. Video encoder 200 and video decoder 300 may also support asymmetric partitioning for PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-prediction.

[0153] In examples where the mode selector does not further partition the CU into PUs, each CU may be associated with a luminance coding block and corresponding chrominance coding blocks. As stated above, the size of the CU may refer to the size of the luminance coding unit of the CU. Video encoder 200 and video decoder 120 may support CU sizes of 2N×2N, 2N×N, or N×2N.

[0154] For other video coding techniques, such as intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, as a few examples, the mode selection module 202, through the respective modules associated with the coding techniques, generates a prediction block for the current encoded block. In some examples, such as palette mode coding, mode selector 202 may not generate a prediction block, but instead generate syntax elements indicating how the block is reconstructed based on the selected palette. In such modes, mode selector 202 may provide these syntax elements to entropy encoding module 220 for encoding.

[0155] As described above, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. Then, the residual generation module 204 generates a residual block for the current block. To generate a residual block, residual generation module 204 calculates the sample-to-sample differences between the prediction block and the current block.

[0156] The transform processing module 206 applies one or more transforms to the residual block to generate a transform coefficient block (referred to herein as a “transform coefficient block”). Transform processing module 206 may apply various transforms to the residual block to generate a block of transform coefficients. For example, transform processing module 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve (KLT) transform, or a conceptually similar transform to the residual block. In some examples, the transform processing module 206 may perform multiple transforms to the residual block, such as a primary transform and a secondary transform such as a rotation transform. In some examples, transform processing module 206 does not apply transforms to the residual block.

[0157] Quantization unit 208 may quantize the transform coefficients in the transform coefficient block to generate a block of quantized transform coefficients. Quantization module 208 may quantize the transform coefficients of a block of transform coefficients in accordance with a quantization parameter (QP) value associated with the current block. Video encoder 200 (eg, via mode selector 202) may adjust the amount of quantization applied to transform coefficient blocks associated with the current block by adjusting the QP value associated with the CU. The quantization may introduce loss of information, and thus the quantized transform coefficients may have lower accuracy than the original transform coefficients generated by the transform processing unit 206 .

[0158] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transforms to the block of quantized transform coefficients, respectively, to recover the residual block from the block of transform coefficients. Restoration module 214 may generate a reconstructed block corresponding to the current block (although possibly with some degree of distortion) based on the reconstructed residual block and the prediction block generated by mode selection module 202 . For example, reconstruction module 214 may add samples of the reconstructed residual block to corresponding samples from the prediction block generated by mode selection module 202 to form a reconstructed block.

[0159] Filtering module 216 may perform one or more filtering operations on the recovered blocks. For example, filter module 216 may perform deblocking operations to reduce deblocking artifacts along the edges of the CU. For example, the filter module 216 may be configured to compare the sample change amount due to the deblocking filtering with the cutoff value, and in response to the amount of sample change due to the deblocking filtering greater than the first cutoff value, change the cutoff value instead of the change amount. sampling due to deblocking filtering. That is, the filter module 216 may be configured to limit the amplitude of the sample change amount. The operations of filter module 216 may be omitted in some examples.

[0160] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in examples where the operations of the filter module 216 are not performed, the recovery module 214 may store the reconstructed blocks in the DPB 218. In the examples where the operations of the filter module 216 are performed, the filter module 216 may store the filtered reconstructed blocks in DPB 218. Motion estimation module 222 and motion compensation module 224 may extract a reference picture from DPB 218 formed from the reconstructed (and potentially filtered) blocks to inter-predict blocks of subsequent encoded pictures. In addition, the intra-prediction module 226 may use the reconstructed blocks in the DPB 218 of the current picture to intra-predict other blocks in the current picture.

[0161] In general, entropy encoding module 220 may entropy encode syntax elements received from other functional components of video encoder 200. For example, entropy encoding module 220 may entropy encode blocks of quantized transform coefficients from quantization module 208 . As another example, entropy encoding unit 220 may entropy encode prediction syntax elements (eg, motion information for inter prediction or intra mode information for intra prediction) from mode selection unit 202 . Entropy encoding module 220 may perform one or more entropy encoding operations on syntax elements, which are another example of video data, to generate entropy encoded data. For example, entropy encoding unit 220 may perform a context adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable to variable length (V2V) coding operation, a syntax-based context adaptive binary arithmetic coding (SBAC) operation, an entropy coding operation with probability interval partitioning (PIPE), an exponential Golomb encoding operation, or another type of entropy data encoding operation. In some examples, entropy encoding module 220 may operate in bypass mode where the syntax elements are not entropy encoded.

[0162] The video encoder 200 may output a bitstream that includes the entropy encoded syntax elements needed to reconstruct slice or picture blocks. In particular, entropy encoding unit 220 may output a bit stream.

[0163] The operations described above are described with respect to a block. Such description is to be understood as operations for the luminance coding block and/or chrominance coding blocks. As described above, in some examples, the luminance coding block and the chrominance coding blocks are the luminance and chrominance components of the CU. In some examples, the luma coding block and the chrominance coding blocks are the luma and chrominance components of the PU.

[0164] In some examples, operations performed on a luminance coding block do not need to be repeated for chrominance coding blocks. As one example, the operations for identifying a motion vector (MV) and a reference picture for a luminance coding block need not be repeated for identifying an MV and a reference picture for chrominance blocks. Rather, the MV for the luminance coding block may be scaled to determine the MV for the chrominance blocks, and the reference picture may be the same. As another example, the intra-prediction process may be the same for the luminance coding block and the chrominance coding blocks.

[0165] FIG. 7 is a block diagram illustrating an exemplary video decoder 300 that may perform the techniques of the present disclosure. Fig. 7 is provided for purposes of explanation and does not limit the techniques broadly illustrated and described in this disclosure. For purposes of explanation, the present disclosure describes a video decoder 300 in accordance with JEM and HEVC techniques. However, the techniques of the present disclosure may be performed by video coding devices that are configured in accordance with other video coding standards.

[0166] In the example of FIG. 7, the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filtering unit 312, and a decoded picture buffer 314. (DPB). The prediction processing module 304 includes a motion compensation module 316 and an intra prediction module 318 . The prediction processing module 304 may include additional modules for performing prediction in accordance with other prediction modes. As examples, prediction processing module 304 may include a palette module, an intrablock copy module (which may form part of motion compensation module 316), an affine module, a linear model (LM) module, or the like. In other examples, video decoder 300 may include more, less, or other functional components.

[0167] The CPB memory 320 may store video data, such as an encoded video bitstream, for decoding by components of the video decoder 300. The video data stored in the CPB memory 320 may be obtained, for example, from a computer-readable medium 110 (FIG. 1). CPB memory 320 may include a CPB that stores encoded video data (eg, syntax elements) from an encoded video bitstream. In addition, the CPB memory 320 may store video data other than the encoded picture syntax elements, such as temporary data representing outputs from various modules of the video decoder 300. DPB 314 typically stores decoded pictures that video decoder 300 can output and/or use as video reference data when decoding subsequent data or pictures of the encoded video bitstream. CPB memory 320 and DPB 314 may be formed by any of a variety of memory devices such as DRAM, including SDRAM, MRAM, RRAM, or other types of memory devices. The CPB memory 320 and DPB 314 may be provided by the same memory device or by separate memory devices. In various examples, CPB memory 320 may be on-chip with other components of video decoder 300, or off-chip with respect to these components.

[0168] Additionally or alternatively, in some examples, video decoder 300 may retrieve encoded video data from memory 120 (FIG. 1). That is, memory 120 may store data as discussed above for CPB memory 320. Similarly, memory 120 may store instructions to be executed by video decoder 300 when some or all of the functionality of video decoder 300 is implemented in software to be executed by video decoder 300 processing circuits.

[0169] The various modules shown in FIG. 7 are illustrated to assist in understanding the operations performed by the video decoder 300. Modules can be implemented as fixed function circuits, programmable circuits, or a combination of the two. Like FIG. 6, fixed functional circuits refer to circuits that provide certain functionality and are pre-configured for operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, the programmable circuits may execute software or firmware that causes the programmable circuits to operate in the manner specified by the instructions of the software or firmware. Fixed function circuits can execute program instructions (for example, accept parameters or output parameters), but the types of operations that fixed function circuits perform are usually immutable. In some examples, one or more modules may be separate circuit blocks (fixed functional blocks or programmable blocks), and in some examples, one or more modules may be integrated circuits.

[0170] The video decoder 300 may include ALUs, EFUs, digital circuits, analog circuits, and/or programmable cores formed by programmable circuits. In instances where the operations of the video decoder 300 are performed by software running on programmable circuits, the on-chip (on-chip) or off-chip memory may store instructions (eg, object code) of the software that the video decoder 300 receives and executes.

[0171] The entropy decoding module 302 may receive the encoded video data from the CPB and entropy decode the video data to reproduce the syntax elements. Prediction processing module 304, inverse quantization module 306, inverse transform processing module 308, reconstruction module 310, and filter module 312 may generate decoded video data based on syntax elements extracted from the bitstream.

[0172] In general, video decoder 300 reconstructs a picture on a block-by-block basis. The video decoder 300 may perform the recovery operation on each block individually (where the block that is being recovered, ie, decoded, at the current time may be referred to as the “current block”).

[0173] Entropy decoding module 302 can entropy decode syntax elements defining quantized transform coefficients of a block of quantized transform coefficients, as well as transform information such as quantization parameter (QP) indication(s) and/or transform mode. Inverse quantization module 306 may use the QP associated with the block of quantized transform coefficients to determine the degree of quantization and, similarly, the degree of inverse quantization to be applied by inverse quantization module 306 . Inverse quantizer 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. Here, inverse quantizer 306 may generate a transform coefficient block including transform coefficients.

[0174] After inverse quantizer 306 has generated a transform coefficient block, inverse transform processing module 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, inverse transform processing unit 308 may apply an inverse DCT, inverse integer transform, inverse Karhunen-Loeve (KLT), inverse rotation transform, inverse directional transform, or other inverse transform to the block of transform coefficients.

[0175] In addition, the prediction processing unit 304 generates a prediction block in accordance with the prediction information syntax elements that have been entropy decoded by the entropy decoding unit 302 . For example, if the prediction information syntax elements indicate that the current block is inter-predicted, motion compensation module 316 may generate a prediction block. In this case, the prediction information syntax elements may indicate a reference picture in the DPB 314 from which the reference block should be retrieved, as well as a motion vector identifying the location of the reference block in the reference picture relative to the location of the current block in the current picture. Motion compensation module 316 may, in principle, perform the inter-prediction process in a manner that is essentially the same as that described with respect to motion compensation module 224 (FIG. 6).

[0176] As another example, if the transformation syntax elements indicate that the current block is intra-predicted, intra-prediction module 318 may generate a prediction block in accordance with the intra-prediction mode indicated by the transformation syntax elements. Again, intra-prediction module 318 can, in principle, perform the intra-prediction process in a manner substantially similar to that described with respect to intra-prediction module 226 (FIG. 6). Intra-prediction module 318 may retrieve adjacent sample data for the current block from DPB 314.

[0177] Restoration module 310 may restore the current block using the transform block and the residual block. For example, recovery module 310 may add residual block samples to corresponding prediction block samples to recover the current block.

[0178] Filtering module 312 may perform one or more filtering operations on the recovered blocks. For example, filter module 312 may perform deblocking operations to reduce deblocking artifacts along the edges of the reconstructed blocks. For example, the filter module 312 may be configured to compare the amount of change to sample due to the deblocking filtering with the cutoff value, and in response to the amount of change to sample due to the deblocking filtering greater than the first cutoff value, change the value by the value clipping, instead of the amount of change for the sample due to deblocking filtering. That is, the filter module 312 may be configured to limit the amplitude of the amount of change for the sample. The operations of filter module 312 need not be performed in all examples.

[0179] The video decoder 300 may store the recovered blocks in the DPB 314. For example, in cases where the operations of the filter module 312 are not required, the recovery module 310 may store the recovered blocks in the DPB 314. In examples where the operations of the filter module 312 are needed, the module 312 filtering may store the filtered reconstructed blocks in DPB 314. As discussed above, DPB 314 may provide reference information such as current picture samples for intra prediction and previously decoded pictures for subsequent motion compensation to prediction processing module 304 . Moreover, video decoder 300 may output decoded pictures (eg, decoded video) from DPB 314 for subsequent presentation on a display device, such as display device 118 in FIG. 1.

[0180] FIG. 8 shows an exemplary implementation of the filter module 312 of FIG. 7. Filter module 216 in FIG. 6 may be implemented in the same or similar manner. Filter modules 216 and 312 may perform the techniques of the present disclosure, possibly in conjunction with other components of video encoder 200 or video decoder 300. In the example in FIG. 8, the filter module 312 includes a deblocking filter 342, a SAO filter 344, and an ALF/GALF filter 346. The SAO filter 344 may, for example, be configured to determine offset values for block samples. ALF/GALF 346 may likewise filter video data blocks using an adaptive loop filter and/or geometric adaptive loop filtering.

[0181] Filter module 312 may include fewer filters and/or may include additional filters. Additionally, the specific filters shown in FIG. 8 may be implemented in a different order. Other loop filters (either in the encoding loop or after the encoding loop) may also be used to smooth out pixel transitions or otherwise improve video quality. The filtered reconstructed video blocks output by filter module 312 may be stored in DPB 314 which stores reference pictures used for subsequent motion compensation. DPB 314 may be part of, or separate from, additional memory that stores decoded video for later presentation on a display device, such as display device 118 in FIG. 1.

[0182] The video decoder 300, such as the deblocking filter 342 of the filtering module 312, may be configured to receive the first reconstructed block of video data and apply deblocking filtering to the first restored block in accordance with any of the techniques or combinations of techniques described in this disclosure. . To obtain the first reconstructed video data block, the video decoder 300 may be configured to determine a video data prediction block and add the residual video data block to the video data prediction block. To apply deblocking filtering to the first reconstructed block, video decoder 300 may apply deblocking filtering to samples located on the boundary of the first reconstructed video data block and the second reconstructed video data block. To apply the deblocking filtering to the first reconstructed block, the video decoder 300 determines the filter strength for the deblocking filtering.

[0183] In order to apply the deblocking filtering to the first reconstructed block, the video decoder 300 may select a filter for the deblocking filtering. To select a filter for deblocking filtering, video decoder 300 may select a filter based on a block length orthogonal to a boundary to be filtered. To select a filter for deblocking filtering, video decoder 300 selects a filter based on a block length aligned with the boundary to be filtered. To select a filter for deblocking filtering, video decoder 300 may select a filter from a plurality of deblocking filters with different lengths. To select a filter for deblocking filtering, video decoder 300 selects a filter from a plurality of deblocking filters with different filter supports.

[0184] The video decoder 300 may be configured to determine a confidence factor and determine the amount of sample change based on the confidence factor. Video decoder 300 may apply deblocking filtering to the first reconstructed block by changing the sample of the first reconstructed block by the sample change amount. The video decoder 300 may determine the sample change amount based on the confidence factor by limiting the sample change amount to a range of values. The confidence factor may, for example, be an interval.

[0185] The video decoder 300 may be configured to determine a second video data prediction block; add a second residual video data block to a second video data prediction block to determine a second reconstructed video data block that is adjacent to the first reconstructed block; determine a first quantization parameter for the residual block; determine a second quantization parameter for the second residual block; apply deblocking filtering to the first reconstructed block based on the first quantization parameter; and apply deblocking filtering to the second reconstructed block based on the second quantization parameter.

[0186] To apply deblocking filtering to the first reconstructed block based on the first quantization parameter, the video decoder 300 may determine a sample change amount for sampling the first reconstructed block based on the first quantization parameter, and apply deblocking filtering to the second reconstructed block based on the second parameter. quantization by determining a sample change amount for sampling the second reconstructed block based on the second quantization parameter. To determine the sample change amount for sampling the first reconstructed block based on the first quantization parameter, the video decoder 300 may limit the sample change amount for sampling the first reconstructed block to a first range of values, and determine the sample change amount for sampling the second reconstructed block based on the second quantization parameter by limiting a sample change amount for sampling the second reconstructed block to a second range of values that is different from the first range of values.

[0187] To apply the deblocking filtering to the first reconstructed block, the video decoder 300 may select deblocking filter parameters for the deblocking filtering. The deblocking filter parameters may, for example, include one or more of filter type, filter length, clipping parameters, or skip space spacing. To select deblocking filter parameters for the deblocking filtering, video decoder 300 may select deblocking filter parameters based on one or more transform properties or transform coefficients for the residual block.

[0188] To apply deblocking filtering to the first reconstructed block, video decoder 300 may determine values for unavailable samples and apply deblocking filtering using the determined values. To determine values for unavailable samples, video decoder 300 may perform an extrapolation process and/or a padding process. The video decoder 300 may determine the unavailable samples as unavailable based on the block response of the first reconstructed block.

[0189] The video decoder 300, such as the deblocking filter 342 of the filter module 312, may be configured to obtain a first reconstructed block of video data, the first reconstructed block including a first sample with a first value; apply deblocking filtering to the first reconstructed block to create a deblocked filtered block; determine a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; compare the first cutoff value with a change amount of the first sample due to the deblocking filtering; in response to a change amount of the first sample due to the deblocking filtering greater than the first cutoff value, changing the first value by the first cutoff value to determine the first filtered value for the first sample; and outputting the deblocked filtered video data block, wherein in the deblocked filtered block, the first sample has the first filtered value.

[0190] The first filtered value may be equal to the first value plus the first cutoff value, or may be equal to the first value minus the first cutoff value. The video decoder 300 may determine the first cutoff value for the first sample further based on the deblocking filtering mode for the first reconstructed block.

[0191] The first reconstructed block may include a second sample with a second value, and the video decoder 300 may determine a second cutoff value for the second sample based on the location of the second sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block and in response to the magnitude change the second sample due to deblocking filtering less than the second cutoff value, change the second value by the change amount of the second sample to determine the second filtered value for the second sample. In such an example, in the deblocked filtered block, the second sample has the second filtered value.

[0192] If the distance between the first sample and the boundary of the first reconstructed block is equal to the distance between the second sample and the boundary of the first reconstructed block, then the video decoder 300 may set the first clipping value to be equal to the second clipping value for the second sample. If the distance between the first sample and the first reconstructed block boundary is different from the distance between the second sample and the first reconstructed block boundary, then the video decoder 300 may set the first clipping value to be different from the second clipping value.

[0193] Based on the size of the first reconstructed block, video decoder 300 may determine that the deblocking filter is applied only to samples that are six samples or less away from the boundary of the first reconstructed block. Based on the size of the first reconstructed block, video decoder 300 may determine that the deblocking filter is applied to samples that are two samples or less away from the boundary of the first reconstructed block.

[0194] The first clipping value may decrease as the distance from the boundary of the first reconstructed block decreases. For example, in response to determining that the deblocking filter is applied to samples within 6 rows or columns of the first reconstructed block boundary, and in response to the first sample being adjacent to the first reconstructed block boundary, video decoder 300 may determine that the first the cutoff value is 6. In response to determining that the deblocking filter is applied to samples within 6 rows or columns of the first reconstructed block boundary, and in response to the first sample being one sample away from the first reconstructed block boundary, the video decoder 300 may determine that the first cutoff value is 5. In response to determining that the deblocking filter is applied to samples within 6 rows or columns of the first reconstructed block boundary, and in response to the first sample being two samples away from the first reconstructed boundary block, the video decoder 300 may determine that the first cutoff value is 4.

[0195] In response to determining that the deblocking filter is applied to samples within 6 rows or columns of the first reconstructed block boundary, and in response to the first sample being three samples away from the first reconstructed block boundary, video decoder 300 may determine that the first cutoff value is 3. In response to determining that the deblocking filter is applied to samples within 6 rows or columns of the first reconstructed block boundary, and in response to the first sample being four samples away from the first reconstructed block boundary, video decoder 300 may determine that the first clipping value is 2. In response to determining that the deblocking filter is applied to samples within 6 rows or columns of the first reconstructed block boundary, and in response to the first sample being five samples away from the boundary of the first recovered block, video decoder 300 may determine that the first cutoff value is 1.

[0196] In response to determining that the deblocking filter is applied to samples within 2 rows or columns of the first reconstructed block boundary, and in response to the first sample being adjacent to the first reconstructed block boundary, video decoder 300 may determine that the first cutoff value is 6. In response to determining that the deblocking filter is applied to samples within 2 rows or columns of the first reconstructed block boundary, and responding that the first sample is one sample away from the first reconstructed block boundary, the video decoder 300 may determine that the first cutoff value is 4. In response to determining that the deblocking filter is applied to samples within 2 rows or columns of the first reconstructed block boundary, and in response to the first sample being two samples away from the first reconstructed boundary block, video decoder 300 may determine that the first cutoff value is 2.

[0197] Video decoder 300 may also be configured to obtain a second reconstructed block of video data, wherein the second reconstructed block includes a second sample with a second value, wherein the second reconstructed block shares a boundary with the first reconstructed block, and wherein the first reconstructed block and the second restored block have different sizes; apply deblocking filtering to the second deblocked block to determine a second deblocking filtered block; determine a second cutoff value for the second sample based on the location of the second sample relative to the boundary of the second reconstructed block and based on the size of the second reconstructed block; responsive to a change amount of the first sample due to the deblocking filtering greater than the second cutoff value, changing the first value by the second cutoff value to determine a second filtered value for the second sample; and outputting a second deblocked filtered video data block, wherein, in the second deblocking filtered block, the second sample has the second filtered value. The video decoder 300 may apply the deblocking filtering to the first deblocked block to determine the first deblocking filtered block that contains the filtering of the samples in the first deblocked block with the first filter of the first length, and apply the deblocking filtering to the second deblocked block to determine the second deblocked block. deblocking the block, which comprises filtering the samples in the second reconstructed block with a second filter of a second length that is different from the first length.

[0198] FIG. 9 is a flowchart illustrating an exemplary method for encoding a current block. The current block may include the current CU. While the description has been made with respect to the video encoder 200 (FIGS. 1 and 6), it should be understood that other devices may be configured to perform a method similar to that shown in FIGS. 9.

[0199] In this example, video encoder 200 initially predicts the current block (350). For example, video encoder 200 may generate a prediction block for the current block. Video encoder 200 may then calculate a residual block for the current block (352). To calculate the residual block, video encoder 200 may calculate the difference between the original, uncoded block and the prediction block for the current block. Video encoder 200 may then transform and quantize the coefficients of the residual block (354). Video encoder 200 may then scan the quantized transform coefficients of the residual block (356). During scanning or after scanning, video encoder 200 may entropy encode the coefficients (358). Video encoder 200 may encode the coefficients using CAVLC or CABAC. Video encoder 200 may then output entropy coded data for the block coefficients (360).

[0200] In FIG. 10 is a flowchart illustrating an exemplary method for decoding the current block. The current block may contain the current CU. Although the description has been made with respect to the video decoder 300 (FIGS. 1 and 7), it should be understood that other devices may be configured to perform a method similar to that shown in FIGS. 10.

[0201] The video decoder 300 may receive entropy coded data for the current block, such as entropy coded prediction information and entropy coded data for residual block coefficients corresponding to the current block (370). Video decoder 300 may entropy decode the entropy encoded data to determine prediction information for the current block and recover coefficients of the residual block (372). The video decoder 300 may predict the current block (374), eg, using an intra- or inter-prediction mode, as indicated by the prediction information for the current block, to compute a prediction block for the current block. Video decoder 300 may then backscan the reconstructed coefficients (376) to create a block of quantized transform coefficients. Video decoder 300 may then inversely quantize and inversely transform the coefficients to form a residual block (378). Video decoder 300 may eventually decode the current block by combining the prediction block and the residual block (380). After combining the prediction block and the residual block to generate a reconstructed block, video decoder 300 may apply one or more filters (eg, deblocking, SAO, and/or ALF/GALF) to the unfiltered reconstructed block to generate a filtered reconstructed block (382).

[0202] In FIG. 11 is a flowchart illustrating an exemplary video decoding technique described in the present disclosure. The procedures according to FIG. 11 will be described with reference to an exemplary video decoder such as, but not limited to, video decoder 300. In some cases, the techniques of FIG. 11 may be performed by a video encoder, such as a video encoder 200, as part of the video encoding process, in which case the exemplary video decoder corresponds to a decoding loop (e.g., inverse quantization module 210, inverse transform processing module 212, reconstruction module 214, and filter module 216) encoder 200 video.

[0203] In the example of FIG. 11, the video decoder receives the first recovered video data block, which includes the first sample with the first value (400). The video decoder applies deblocking filtering to the first reconstructed block to create a deblocking filtered block (402). The video decoder determines the first clipping value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block (404). The video decoder 300 compares the first clipping value with the amount of change in the first sample due to the deblocking filtering (406). In response to a first sample change amount due to deblocking filtering greater than the first cutoff value (408, YES), the video decoder modifies the first value by the first cutoff value to determine the first filtered value for the first sample (410). In response to a first sample change amount due to the deblocking filtering not greater than the first cutoff value (408, NO), the video decoder modifies the first value by the first sample change amount due to the deblocking filtering (412). The video decoder may, for example, store the first filtered value equal to the filtered value determined by the deblocking filtering, rather than setting the first filtered value equal to the first value plus or minus the clipping value.

[0204] It should be borne in mind that, depending on the example, some actions or events of any of the techniques described here may be performed in a different sequence, may be added, combined, or excluded altogether (for example, not all of the described actions or events are necessary for practical implementation methods). In addition, in some examples, actions or events may be performed simultaneously, such as through multi-threading, interrupt handling, or multiple processors, rather than sequentially.

[0205] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware processing module. Computer-readable media may include computer-readable media that corresponds to a tangible medium, such as a storage medium, or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another, such as in accordance with a communication protocol. Thus, computer-readable media can typically correspond to (1) tangible computer-readable storage media that is durable, or (2) a communication medium such as a signal or carrier wave. Storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described in this disclosure. The computer program product may include a computer readable medium.

[0206] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly defined as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable , fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the media definition. However, it should be understood that computer-readable storage media and storage media do not include connections, carrier waves, signals, or other temporary media, but are instead directed to durable, tangible storage media. The magnetic disc (disk) and optical disc (disc) as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disc, and Blu-ray disc, where magnetic discs usually reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer readable media.

[0207] The instructions may be executed by one or more processors, such as one or more DSPs, general purpose microprocessors, ASICs, FPGAs, or other equivalent integrated circuits or discrete logic circuits. Accordingly, the term "processor", as used here, may refer to any of the above structures or any other structure suitable for implementing the techniques described here. In addition, in some aspects, the functionality described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding or embedded in an integrated codec. In addition, the techniques may be fully implemented in one or more circuits or logic elements.

[0208] The techniques of the present disclosure may be implemented in a wide variety of devices or devices, including a wireless telephone, an integrated circuit (IC), or a stack of integrated circuits (eg, a chipset). Various components, modules, or blocks are described in this disclosure with emphasis on the functional aspects of devices configured to perform the disclosed techniques, but not necessarily requiring implementation by different hardware modules. Rather, as described above, various modules may be combined into a codec hardware module or provided with a set of cooperating hardware modules, including one or more processors, as described above, in association with suitable software and/or firmware.

[0209] Various examples have been described. These and other examples are within the scope of the following claims.

Claims (90)

1. A method for decoding video data, comprising the steps of: receiving a first reconstructed video data block, the first reconstructed block including a first discrete sample with a first value; applying deblocking filtering to the first reconstructed block to create a deblocked filtered block; determining a first cutoff value for the first discrete sample based on the location of the first discrete sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; comparing the first clipping value with the amount of change of the first discrete sample due to deblocking filtering; in response to the change amount of the first sample due to the deblocking filtering being greater than the first cutoff value, modifying the first value by the first cutoff value to determine the first filtered value for the first sample; And outputting the deblocked filtered video data block, wherein in the deblocked filtered block the first discrete sample has the first filtered value. 2. The method of claim 1, wherein the first filtered value is equal to the first value plus the first cutoff value. 3. The method of claim 1, wherein the first filtered value is equal to the first value minus the first cutoff value. 4. The method of claim 1, further comprising determining a first cutoff value for the first sample further based on the deblocking filtering mode for the first recovered block. 5. The method of claim 1, wherein the first recovered block includes a second discrete sample with a second value, the method further comprising: determining a second cutoff value for the second sample based on the location of the second sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; in response to the change amount of the second sample due to the deblocking filtering being less than the second cutoff value, modifying the second value by the change amount of the second sample to determine a second filtered value for the second sample; And wherein in the deblocked filtered block, the second discrete sample has a second filtered value. 6. The method of claim 5, wherein the distance between the first sample and the boundary of the first reconstructed block is equal to the distance between the second sample and the boundary of the first reconstructed block, wherein the first cutoff value is equal to the second cutoff value for the second discrete sample. 7. The method of claim 5, wherein the distance between the first sample and the boundary of the first reconstructed block is different from the distance between the second sample and the boundary of the first reconstructed block, wherein the first cutoff value is different from the second cutoff value. 8. The method of claim 1, further comprising determining, based on the size of the first reconstructed block, that the deblocking filter is applicable only to samples that are six or less discrete samples away from the boundary of the first reconstructed block. 9. The method of claim 8, further comprising, in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary, and in response to that the first discrete sample is adjacent to the boundary of the first reconstructed block, it is determined that the first clipping value is 6. 10. The method of claim 8, further comprising, in response to determining that the deblocking filter is applicable to the discrete samples within 6 rows or columns of the first reconstructed block boundary, and in response to the fact that the first discrete sample on one sample away from the boundary of the first reconstructed block, it is determined that the first cutoff value is 5. 11. The method of claim 8, further comprising, in response to determining that the deblocking filter is applicable to the discrete samples within 6 rows or columns of the first reconstructed block boundary, and in response to the fact that the first discrete sample is on two discrete samples away from the boundary of the first reconstructed block determine that the first cutoff value is 4. 12. The method of claim 8, further comprising, in response to determining that the deblocking filter is applicable to the discrete samples within 6 rows or columns of the first reconstructed block boundary, and in response to the fact that the first discrete sample on three discrete samples away from the boundary of the first reconstructed block determine that the first cutoff value is 3. 13. The method of claim 8, further comprising, in response to determining that the deblocking filter is applicable to the discrete samples within 6 rows or columns of the first reconstructed block boundary, and in response to the fact that the first discrete sample is on four discrete samples away from the boundary of the first reconstructed block determine that the first cutoff value is 2. 14. The method of claim 8, further comprising, in response to determining that the deblocking filter is applicable to the discrete samples within 6 rows or columns of the first reconstructed block boundary, and in response to the fact that the first discrete sample on five discrete samples away from the boundary of the first reconstructed block determine that the first cutoff value is 1. 15. The method of claim 1, further comprising, based on the size of the first reconstructed block, determining that the deblocking filter is applicable to samples that are two or less samples away from the boundary of the first reconstructed block. 16. The method of claim 15, further comprising, in response to determining that the deblocking filter is applicable to discrete samples within 2 rows or columns of a first reconstructed block boundary, and in response to that the first discrete sample is adjacent to the boundary of the first reconstructed block, it is determined that the first clipping value is 6. 17. The method of claim 15, further comprising, in response to determining that the deblocking filter is applicable to discrete samples within 2 rows or columns of a first reconstructed block boundary, and in response to the fact that the first discrete sample is on one sample away from the boundary of the first reconstructed block, it is determined that the first cutoff value is 4. 18. The method of claim 15, further comprising, in response to determining that the deblocking filter is applicable to discrete samples within 2 rows or columns of a first reconstructed block boundary, and in response to that the first discrete sample on two discrete samples away from the boundary of the first reconstructed block determine that the first cutoff value is 2. 19. The method of claim 1, further comprising the steps of: obtaining a second reconstructed block of video data, the second reconstructed block including a second discrete sample with a second value, the second reconstructed block having a common boundary with the first reconstructed block, and the first reconstructed block and the second reconstructed block having different sizes; applying deblocking filtering to the second reconstructed block to determine a second deblocking filtered block; determining a second cutoff value for the second sample based on the location of the second sample relative to the boundary of the second reconstructed block and based on the size of the second reconstructed block; in response to the change amount for the second sample due to the deblocking filtering being greater than the second cutoff value, modifying the second value by the second cutoff value to determine a second filtered value for the second sample; And outputting a second deblocked filtered video data block, wherein in the second deblocking filtered block the second discrete sample has a second filtered value. 20. The method according to claim 19, in which: when applying the deblocking filtering to the first reconstructed block, to determine the first deblocking filtered block, filtering the discrete samples in the first reconstructed block with the first filter of the first length; And applying deblocking filtering to the second reconstructed block to determine a second deblocking filtered block, filtering the discrete samples in the second reconstructed block with a second filter of a second length that is different from the first length. 21. The method of claim 1, wherein the method is performed as part of the video encoding process. 22. A device for decoding video data, comprising: a memory adapted to store video data; one or more processors implemented in circuitry and configured to: obtain a first reconstructed block of video data, the first reconstructed block including a first discrete sample with a first value; apply deblocking filtering to the first reconstructed block to create a deblocked filtered block; determine a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; compare the first clipping value with the amount of change of the first discrete sample due to deblocking filtering; in response to the change amount of the first sample due to the deblocking filtering being larger than the first cutoff value, changing the first value by the first cutoff value to determine the first filtered value for the first sample; output the deblocked filtered video data block, wherein in the deblocked filtered block the first discrete sample has the first filtered value. 23. The apparatus of claim 22, wherein the first filtered value is equal to the first value plus the first cutoff value. 24. The apparatus of claim 22, wherein the first filtered value is equal to the first value minus the first cutoff value. 25. The apparatus of claim 22, wherein the one or more processors are further configured to determine the first cutoff value for the first discrete sample further based on the deblocking filtering mode for the first recovered block. 26. The apparatus of claim 22, wherein the reconstructed block includes a second discrete sample with a second value, and wherein the one or more processors are further configured to: determine a second cutoff value for the second sample based on the location of the second sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; in response to the change amount of the second sample due to the deblocking filtering being less than the second cutoff value, changing the second value by the change amount for the second sample to determine a second filtered value for the second sample; And wherein, in the deblocking filtered block, the second discrete sample has a second filtered value. 27. The apparatus of claim 26, wherein the distance between the first sample and the boundary of the first reconstructed block is equal to the distance between the second sample and the boundary of the first reconstructed block, and wherein the first cutoff value is equal to the second cutoff value for the second discrete sample. 28. The apparatus of claim 26, wherein the distance between the first sample and the first reconstructed block boundary is different from the distance between the second sample and the first reconstructed block boundary, and wherein the first cutoff value is different from the second cutoff value. 29. The apparatus of claim 22, wherein the one or more processors are further configured to determine, based on the size of the first reconstructed block, that the deblocking filter is applicable only to samples that are six or less samples away from the boundary of the first reconstructed block. 30. The apparatus of claim 29, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary and in response to the first sample is adjacent to the boundary of the first reconstructed block, determine that the first cutoff value is 6. 31. The apparatus of claim 29, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary and in response to the first sample is one sample away from the first reconstructed block boundary, determine that the first cutoff value is 5. 32. The apparatus of claim 29, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary and in response to the first bin is two bins away from the first reconstructed block boundary, determine that the first cutoff value is 4. 33. The apparatus of claim 29, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary and in response to the first bin is three bins away from the first reconstructed block boundary, determine that the first cutoff value is 3. 34. The apparatus of claim 29, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary and in response to the first bin is four bins away from the first reconstructed block boundary, determine that the first cutoff value is 2. 35. The apparatus of claim 29, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 6 rows or columns of a first reconstructed block boundary and in response to the first bin is five bins away from the first reconstructed block boundary, determine that the first cutoff value is 1. 36. The apparatus of claim 22, wherein the one or more processors are further configured, based on the size of the first reconstructed block, to determine that the deblocking filter is applicable to samples that are two or less samples away from the boundary of the first reconstructed block. 37. The apparatus of claim 36, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 2 rows or columns of a first reconstructed block boundary and in response to the first sample is adjacent to the boundary of the first reconstructed block, determine that the first cutoff value is 6. 38. The apparatus of claim 36, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 2 rows or columns of a first reconstructed block boundary and in response to the first sample is one sample away from the boundary of the first reconstructed block, determine that the first cutoff value is 4. 39. The apparatus of claim 36, wherein the one or more processors are further configured in response to determining that the deblocking filter is applicable to discrete samples within 2 rows or columns of a first reconstructed block boundary and in response to the first sample is two samples away from the first reconstructed block boundary, determine that the first cutoff value is 2. 40. The apparatus of claim 22, wherein the one or more processors are further configured to: obtaining a second reconstructed block of video data, the second reconstructed block including a second discrete sample with a second value, the second reconstructed block having a common boundary with the first reconstructed block, and wherein the first reconstructed block and the second reconstructed block have different sizes; apply deblocking filtering to the second deblocked block to determine a second deblocked filtered block; determine a second cutoff value for the second sample based on the location of the second sample relative to the boundary of the second reconstructed block and based on the size of the second reconstructed block; in response to the change amount of the second sample due to the deblocking filtering being greater than the second cutoff value, changing the second value by the second cutoff value to determine a second filtered value for the second sample; And outputting a second deblocked filtered video data block, wherein in the second deblocking filtered block, the second sample has a second filtered value. 41. The device according to claim 40, in which: for applying deblocking filtering to the first reconstructed block to determine the first deblocking filtered block, one or more processors are further configured to filter the samples in the first reconstructed block with the first filter of the first length; And to apply deblocking filtering to the second deblocked block to determine a second deblocking filtered block, one or more processors are further configured to filter the discrete samples in the second deblocked block with a second filter of a second length that is different from the first length. 42. The apparatus of claim 22, wherein the apparatus comprises a video encoder configured to decode video data as part of a video encoding process. 43. A computer-readable storage medium that stores instructions that, when executed by one or more processors, cause one or more processors to: obtain a first reconstructed block of video data, wherein the first reconstructed block includes a first discrete sample with a first value; apply deblocking filtering to the first reconstructed block to create a deblocked filtered block; determine a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; compare the first clipping value with the amount of change of the first discrete sample due to deblocking filtering; in response to the change amount of the first sample due to the deblocking filtering being larger than the first cutoff value, changing the first value by the first cutoff value to determine the first filtered value for the first sample; And output the deblocked filtered video data block, wherein in the deblocked filtered block the first discrete sample has the first filtered value. 44. A device for decoding video data, comprising: means for obtaining a first reconstructed block of video data, wherein the first reconstructed block includes a first discrete sample with a first value; means for applying deblocking filtering to the first reconstructed block to create a deblocking filtered block; means for determining a first cutoff value for the first sample based on the location of the first sample relative to the boundary of the first reconstructed block and based on the size of the first reconstructed block; means for comparing the first clipping value with a change amount of the first sample sample due to deblocking filtering; means for changing the first value by the first clip value to determine the first filtered value for the first sample in response to the fact that the amount of change of the first sample due to deblocking filtering is greater than the first clip value; And means for outputting the deblocked filtered video data block, wherein in the deblocked filtered block the first discrete sample has the first filtered value.

Images