JP7412443B2 - Method and apparatus for nonlinear loop filtering - Google Patents

Description

Incorporated by Reference This application claims priority to U.S. Provisional Application No. 62/896,997, “Improved Nonlinear Loop Filtering,” filed on September 6, 2019, filed on July 9, 2020. Claims priority to "Method and Apparatus for Nonlinear Loop Filtering," filed U.S. patent application Ser. No. 16/924,845. The entire contents of which are incorporated herein by reference.

This disclosure generally describes embodiments related to video coding.

The "background" description provided herein is for the purpose of generally providing a context for the disclosure. The work of the currently named inventors, as well as aspects of the description that do not qualify as prior art at the time of filing, are expressly cited as prior art to this disclosure to the extent described in this Background section. It is not even implicitly acknowledged.

Video coding and decoding may be performed using intra picture prediction with motion compensation. Uncompressed digital video may include a series of pictures, each picture having a spatial dimension of, for example, 1920x1080 luma samples and associated chroma samples. The series of pictures may have a fixed or variable picture rate (also informally referred to as "frame rate"), for example 60 pictures per second or 60 Hz. Uncompressed video has significant bit rate requirements. For example, 1080p60 4:2:0 video at 8 bits per sample (1920x1080 luma sample resolution at 60Hz frame rate) requires a bandwidth close to 1.5 Gbit/s. One hour of such video requires over 600 GBytes of storage space.

One purpose of video coding and decoding may be to reduce redundancy in the input video signal through compression. Compression helps reduce the aforementioned bandwidth or storage space requirements by more than two orders of magnitude in some cases. Both lossless and lossy compression, and combinations thereof, can be used. Lossless compression refers to a technique in which an exact copy of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original signal and the reconstructed signal may cause the reconstructed signal to be less than the intended signal. small enough to be useful for applications. In the case of video, lossy compression is widely adopted. The amount of distortion allowed depends on the application. For example, users of certain consumer streaming applications can tolerate higher distortion than users of television publishing applications. The achievable compression ratio may reflect that the higher the acceptable/tolerable distortion, the higher the compression ratio.

Video encoders and decoders may utilize techniques from several broad categories including, for example, motion compensation, transforms, quantization, and entropy coding.

Video codec technology may include a technique known as intra-coding. In intra-coding, sample values are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into blocks of samples. If all blocks of samples are coded in intra mode, the picture may become an intra picture. Intra pictures and their independent derivatives, such as decoder refresh pictures, can be used to reset the decoder state, and thus are used in coded video bitstreams and as the first picture in a video session, or as still images. be able to. Intra-block samples can be subjected to a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction may be a technique that minimizes sample values in the pre-transform domain. In some cases, the smaller the DC value and the smaller the AC coefficient after the transform, the fewer bits are needed for a given quantization step size to represent the block after entropy coding.

Conventional intra-coding, as known for example from MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques e.g. Includes techniques to try. Such techniques are hereinafter referred to as "intra-prediction" techniques. Note that, at least in some cases, intra prediction only uses reference data from the current picture being reconstructed and not from reference pictures.

Intra predictions can exist in various forms. If two or more of such techniques are available for a given video coding technique, the technique in use may be coded in intra-prediction mode. In some cases, a mode may have submodes or parameters, which may be individually coded or included in the mode codeword. Which codewords are used for a given mode/submode/parameter combination can affect the coding efficiency improvement due to intra-prediction, so it is important to consider which codewords are used to convert the codewords into a bitstream. may affect the entropy coding techniques used.

The specific mode of intra prediction is H. Filed under 264, H. H.265 and further improved with newer coding techniques such as Joint Search Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). The predictor block can be formed using adjacent sample values belonging to already available samples. The sample values of neighboring samples are copied into the predictor block according to the direction. References to directions in use may be encoded in the bitstream or predicted themselves.

Referring to FIG. 1A, shown at the bottom right is H. There are 265 33 predictable predictor directions (corresponding to 35 intra modes and 33 angular modes). The point where the arrows converge (101) represents the sample being predicted. The arrow represents the direction in which the samples are predicted. For example, arrow (102) indicates that sample (101) is predicted from one or more samples towards the top right and at an angle of 45 degrees from the horizontal axis. Similarly, arrow (103) indicates that sample (101) is predicted from one or more samples to the bottom left of sample (101) at an angle of 22.5 degrees from the horizontal axis.

Continuing to refer to FIG. 1A, at the top left is shown a square block (104) of 4×4 samples (indicated by a thick dashed line). Square blocks (104) each include 16 samples labeled with "S", its location in the Y dimension (eg, row index), and its location in the X dimension (eg, column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in block (104) in both the Y and X dimensions. Since the block is 4×4 samples in size, S44 is at the bottom right. Additionally, a reference sample following a similar numbering scheme is shown. The reference sample is labeled with R, its Y position (eg, row index) and X position (column index) relative to block (104). H. 264 and H. In both H.265, the predicted samples are adjacent to the block being reconstructed. Therefore, there is no need to use negative values.

Intra-picture prediction can work by copying reference sample values from appropriate neighboring samples in a signaled prediction direction. For example, if the encoded video bitstream matches the arrow (102) for this block (i.e., samples are predicted from one or more predicted samples to the top right and at an angle of 45 degrees from horizontal) It is assumed that the prediction direction includes signaling indicating the prediction direction. In this case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

In certain cases, the values of multiple reference samples can be combined, for example by interpolation, to calculate the reference sample, especially if the directions are not evenly divided by 45 degrees.

As video coding technology has evolved, the number of predictable directions has increased. H. 264 (2003) could represent nine different directions. H. 265 (2013), JEM/VVC/BMS can support up to 65 directions at the time of disclosure. Experiments are performed to identify the most likely direction, and certain techniques in entropy coding are used to represent the likely direction in a small number of bits, accepting a certain penalty for the less likely direction. Additionally, the direction itself may be predicted from neighboring directions used in neighboring, already decoded blocks.

FIG. 1B shows a schematic diagram (180) showing 65 intra-prediction directions by JEM to illustrate the increasing number of prediction directions over time.

The mapping of intra-prediction direction bits in a coded video bitstream representing direction can be different for different video coding techniques, and can also be used, for example, by a simple direct mapping of prediction direction to intra-prediction mode to codeword. can range from complex adaptive schemes including most likely modes and similar techniques. However, in all cases, there may be certain directions that are statistically less likely to occur in video content than other certain directions. Since the purpose of video compression is redundancy reduction, in a properly functioning video coding technique these less likely directions are represented by a greater number of bits than more likely directions.

Motion compensation may be a lossy compression technique that spatially compresses blocks of sample data from a previously reconstructed picture or a portion thereof (the reference picture) in the direction indicated by a motion vector (hereinafter "MV"). may be related to the technique used to predict the newly reconstructed picture or picture portion after shifting to . In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV can have two dimensions, X and Y, or three dimensions, with the third dimension indicating the reference picture in use (the latter can indirectly be the temporal dimension).

In some video compression techniques, the MV applicable to a particular region of sample data is derived from other MVs, e.g. samples that are spatially adjacent to the region being reconstructed and earlier in the decoding order than that MV. It can be predicted from MVs related to different areas of data. Doing so can significantly reduce the amount of data required to encode the MV, thereby removing redundancy and enhancing compression. For example, when encoding an input video signal derived from a camera (referred to as "natural video"), there is a statistical chance that a region larger than the region to which a single MV is applied will move in a similar direction. Therefore, MV prediction can work effectively. Therefore, in some cases, similar motion vectors derived from the MVs of adjacent regions can be used for prediction. As a result, the detected MVs for a particular region are similar or identical to the MVs predicted from the surrounding MVs, and conversely, after entropy coding, fewer bits are required than when directly coding the MVs. can be expressed. In some cases, MV prediction may be an example of lossless compression of a signal (i.e., "MV") derived from the original signal (i.e., "sample stream"). In other cases, the MV prediction itself may be irreversible, for example due to rounding errors when computing the predictor from several surrounding MVs.

H. H.265/HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016) describes various MV prediction mechanisms. H. Among the many MV prediction mechanisms provided by H.265, the one described here is a technique hereinafter referred to as "spatial merging."

Referring to FIG. 2, the current block (201) may include samples found in a motion search process by the encoder to be predictable from a spatially shifted previous block of the same size. Instead of directly encoding that MV, one The MV can be derived from the metadata associated with the above reference pictures, for example from the latest reference picture (in decoding order). H. In H.265, MV prediction can use predictors from the same reference pictures that neighboring blocks are using.

Aspects of the present disclosure provide methods and apparatus for video encoding/decoding. In some examples, an apparatus for video decoding includes processing circuitry. The processing circuit is capable of decoding coding information of coding blocks in pictures of a coded video sequence. The coding information may indicate a clipping index m, and the clipping index m indicates a clipping value of a filter applied to the coding block. The processing circuit may determine the clipping value associated with the clipping index. The clipping value may be based on a multiplication of a first function and a second function. The first function depends on the bit depth B and is independent of the clipping index m, and the second function depends on the clipping index m and is independent of the bit depth B. . The processing circuit may generate a filtered coding block by applying the filter corresponding to the clipping value to the coding block. In one embodiment, the first function is proportional to 2 ^B.

In one embodiment, the clipping value is an integer. The processing circuit may determine the clipping value based on 2 ^B 2 ^−αm . The first function is 2 ^B and the second function is 2 ^{- αm} , where α is a constant value associated with the strength of the filter.

In one embodiment, the filter is a non-linear adaptive loop filter that includes a clipping function that depends on the clipping value.

In one example, the clipping index m is one of 0, 1, 2, and 3.

In one embodiment, the constant value α is a first constant value based on the coding block being a luma coding block, and the constant value α is a first constant value based on the coding block being a chroma coding block. It is a constant value of 2. In one example, the first constant value is different than the second constant value. In one example, the first constant value is 2.3 and the second constant value is 2.6.

In one embodiment, the coding information indicating the clipping index m is signaled in an adaptive parameter set (APS) of the picture. The processing circuit may receive a filter set index for the coding block and determine the filter from a plurality of filter sets based on the filter set index.

Aspects of the present disclosure also provide a non-transitory computer-readable medium having instructions stored thereon that, when executed by a computer for video decoding, cause the computer to perform any of the methods for video decoding.

Further features, nature, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

FIG. 2 is a schematic diagram of an exemplary subset of intra prediction modes; It is an explanatory diagram of an example intra prediction direction. FIG. 2 is a schematic diagram of a current block and its surrounding spatial merging candidates in an example; 1 is a schematic block diagram of a communication system (300) according to one embodiment. FIG. 1 is a schematic block diagram of a communication system (400) according to one embodiment. FIG. 1 is a schematic block diagram of a decoder according to one embodiment; FIG. 1 is a schematic block diagram of an encoder according to one embodiment; FIG. FIG. 6 shows a block diagram of an encoder according to another embodiment. FIG. 3 shows a block diagram of a decoder according to another embodiment. 4 illustrates examples of filter shapes according to embodiments of the present disclosure. 5 illustrates an example of subsampled locations used to calculate a slope, according to embodiments of the present disclosure. 5 illustrates an example of subsampled locations used to calculate a slope, according to embodiments of the present disclosure. 5 illustrates an example of subsampled locations used to calculate a slope, according to embodiments of the present disclosure. 5 illustrates an example of subsampled locations used to calculate a slope, according to embodiments of the present disclosure. 2 illustrates an example of a virtual boundary filtering process according to an embodiment of the present disclosure. 2 illustrates an example of a virtual boundary filtering process according to an embodiment of the present disclosure. 2 illustrates an example of symmetric padding operations at virtual boundaries according to embodiments of the present disclosure. 2 illustrates an example of symmetric padding operations at virtual boundaries according to embodiments of the present disclosure. 2 illustrates an example of symmetric padding operations at virtual boundaries according to embodiments of the present disclosure. 2 illustrates an example of symmetric padding operations at virtual boundaries according to embodiments of the present disclosure. 2 illustrates an example of symmetric padding operations at virtual boundaries according to embodiments of the present disclosure. 2 illustrates an example of symmetric padding operations at virtual boundaries according to embodiments of the present disclosure. 2 shows a table illustrating the relationship between clipping values and bit depth B and clipping index m according to an embodiment of the present disclosure. 2 shows a table illustrating the relationship between clipping values and bit depth B and clipping index m according to an embodiment of the present disclosure. 15 shows a flowchart outlining a process (1500) according to one embodiment of the present disclosure. 1 is a schematic diagram of a computer system according to one embodiment. FIG.

FIG. 3 shows a schematic block diagram of a communication system (300) according to one embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other via, for example, a network (350). For example, a communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via a network (350). In the example of FIG. 3, the first pair of terminals (310) and (320) perform unidirectional transmission of data. For example, a terminal device (310) encodes video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) over a network (350). obtain. Encoded video data may be transmitted in the form of one or more encoded video bitstreams. The terminal device (320) may receive coded video data from the network (350), decode the coded video data to recover video pictures, and display the video pictures according to the recovered video data. Unidirectional data transmission is common, such as in media serving applications.

In other examples, the communication system (300) includes a second pair of terminal devices (330) and (340) that perform bidirectional transmission of coded video data, which may occur, for example, during a video conference. In the case of bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) transmits to the other terminal device of the terminal devices (330) and (340) via the network (350). Video data (eg, a stream of video pictures captured by a terminal device) may be encoded for purposes. Each of the terminal devices (330) and (340) can receive the coded video data transmitted by the other terminal device of the terminal devices (330) and (340), and can decode the coded video data. The video picture can be restored and the video picture can be displayed on an accessible display device according to the restored video data.

In the example of FIG. 3, terminal devices (310), (320), (330), and (340) may be shown as a server, a personal computer, and a smartphone, but the principles of the present disclosure are not limited thereto. There isn't. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and/or dedicated video conferencing equipment. Network (350) represents any number of networks that convey coded video data between terminal devices (310), (320), (330) and (340), including, for example, wired and/or wireless communication networks. . The communication network (350) may exchange data over circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and/or the Internet. For purposes of this discussion, the architecture and topology of network (350) may not be important to the operation of this disclosure, unless described below.

FIG. 4 illustrates the deployment of video encoders and video decoders in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, and storage of compressed video on digital media, including CDs, DVDs, memory sticks, and the like.

The streaming system may include a capture subsystem (413) that may include a video source (401), such as a digital camera, that creates a stream of uncompressed video pictures (402). In one example, the stream of video pictures (402) includes samples captured by a digital camera. A stream of video pictures (402), shown in bold to emphasize the high amount of data compared to the encoded video data (404) (or encoded video bitstream), is connected to the video source (401). It may be processed by an electronic device (420) that includes an associated video encoder (403). Video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (or encoded video bitstream (404)), shown with a thin line to emphasize the lower amount of data compared to the stream of video pictures (402), will in the future It can be stored on the streaming server (405) for use. One or more streaming client subsystems, such as client subsystems (406) and (408) of FIG. 4, access a streaming server (405) to obtain a copy (407) of encoded video data (404). and (409) can be searched. Client subsystem (406) may include, for example, a video decoder (410) in electronic device (430). A video decoder (410) decodes the incoming copy (407) of the encoded video data to produce a video picture (410) that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). 411). In some streaming systems, encoded video data (404), (407), and (409) (eg, video bitstreams) may be encoded according to a particular video coding/compression standard. Examples of these standards are ITU-T Recommendation H. 265 included. In one example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

Note that electronic devices (420) and (430) may include other components (not shown). For example, electronic device (420) can include a video decoder (not shown), and electronic device (430) can also include a video encoder (not shown).

FIG. 5 shows a block diagram of a video decoder (510) according to one embodiment of the present disclosure. A video decoder (510) may be included in an electronic device (530). Electronic device (530) can include a receiver (531) (eg, receiving circuitry). Video decoder (510) can be used instead of video decoder (410) in the example of FIG.

The receiver (531) may receive one or more coded video sequences to be decoded by the video decoder (510), and in the same or another embodiment, one coded video sequence at a time. The decoding of each coded video sequence may be independent of other coded video sequences. A coded video sequence may be received from a channel (501), which may be a hardware/software link to a storage device that stores encoded video data. The receiver (531) may receive the encoded video data along with other data, such as encoded audio data and/or auxiliary data streams, which may be transferred to the respective usage entity (not shown). A receiver (531) can separate the coded video sequence from other data. To prevent network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder/parser (520) (hereinafter "parser (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be external to the video decoder (510) (not shown). In still other cases, there is a buffer memory (not shown) external to the video decoder (510), e.g. to prevent network jitter, and also in the video decoder (510), e.g. to handle playback timing. There may be another buffer memory (515) inside. The buffer memory (515) may not be required or may be small when the receiver (531) receives data from a storage/transfer device with sufficient bandwidth and controllability or from an isosynchronous network. Sometimes. For use in best effort packet networks such as the Internet, a buffer memory (515) may be required, and the buffer memory (515) may be relatively large and advantageously may be of adaptive size. , may be at least partially implemented in an operating system or similar element (not shown) external to the video decoder (510).

The video decoder (510) may include a parser (520) that reconstructs symbols (521) from the coded video sequence. These categories of symbols represent information used to manage the operation of the video decoder (510) and, although not an integral part of the electronic device (530), as shown in FIG. ) that may be coupled to a rendering device (512) (eg, a display screen). Control information for the rendering device may be in the form of Supplemental Enhancement Information (SEI messages) or Video Usability Information (VUI) parameter set fragments (not shown). A parser (520) may parse/entropy decode the received coded video sequence. Coding of the coded video sequence may be tailored to a video coding technique or standard and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. A parser (520) may extract a set of subgroup parameters for at least one subgroup of pixels in a video decoder from the coded video sequence based on at least one parameter corresponding to the group. A subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (520) may also extract information such as transform coefficients, quantization parameter values, motion vectors, etc. from the coded video sequence.

A parser (520) may perform entropy decoding/parsing operations on the video sequence received from the buffer memory (515) to create symbols (521).

Reconstruction of symbols (521) may involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter and intra pictures, inter and intra blocks), and other factors. can. Which units participate and how can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not shown for clarity.

In addition to the functional blocks already mentioned, the video decoder (510) can be conceptually subdivided into several functional units as explained below. In actual implementation under commercial constraints, many of these units interact closely with each other and may be at least partially integrated with each other. However, for purposes of explanation of the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

The first unit is a scaler/inverse transform unit (551). The scaler/inverse transform unit (551) receives control information including the transform to use, block size, quantization factor, quantization scaling matrix, etc., and quantized transform coefficients as symbols (521) from the parser (520). do. The scaler/inverse transform unit (551) can output blocks containing sample values that can be input to the aggregator (555).

In some cases, the output samples of the scaler/inverse transform unit (551) are intra-coding blocks, i.e., not using prediction information from a previously reconstructed picture, but from a previously reconstructed part of the current picture. may be related to blocks for which predictive information is available. Such prediction information may be provided by an intra picture prediction unit (552). In some cases, the intra picture prediction unit (552) generates a block of the same size and shape of the block being reconstructed using surrounding already reconstructed information retrieved from the current picture buffer (558). . The current picture buffer (558) buffers, for example, a partially reconstructed current picture and/or a fully reconstructed current picture. The aggregator (555) adds the prediction information generated by the intra prediction unit (552) to the output sample information provided by the scaler/inverse transform unit (551), possibly on a sample by sample basis.

In other cases, the output samples of the scaler/inverse transform unit (551) may relate to inter-coded, potentially motion-compensated blocks. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to retrieve samples used for prediction. After motion compensating the retrieved samples according to the symbols (521) associated with the block, these samples are added to the output of the scaler/inverse transform unit (551) by an aggregator (555) to generate output sample information. (in this case called residual samples or residual signals). The address in the reference picture memory (557) from which the motion compensated prediction unit (553) retrieves the predicted samples is, for example, the address in the reference picture memory (557) from which the motion compensated prediction unit (553) ) can be controlled by the motion vectors available. Motion compensation may also include interpolation of sample values retrieved from reference picture memory (557) when sub-sample accurate motion vectors are in use, motion vector prediction mechanisms, etc.

The output samples of the aggregator (555) are subjected to various loop filtering techniques in a loop filter unit (556). The video compression technique is controlled by parameters made available to the loop filter unit (556) as symbols (521) from the parser (520), contained in the coded video sequence (also referred to as the coded video bitstream). and depending on the meta-information obtained during decoding of the previous part (in decoding order) of the coded picture or coded video sequence, as well as depending on the pre-reconstructed and loop-filtered sample values. In-loop filter techniques may also be included.

The output of the loop filter unit (556) is a sample stream that can be output to a rendering device (512) and stored in a reference picture memory (557) for use in future inter-picture predictions. could be.

Once a particular coded picture is completely reconstructed, it can be used as a reference picture for future predictions. For example, once the coded picture corresponding to the current picture has been completely reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (520)), the current picture buffer (558) 557) and reallocate a new current picture buffer before starting reconstruction of the next coded picture.

The video decoder (510) is an ITU-T Rec. H. The decoding operation may be performed according to a predetermined video compression technique in a standard such as H.265. A coded video sequence is a coded video sequence that conforms to both the syntax of the video compression technology or standard and the profile documented in the video compression technology or standard. It may conform to the syntax specified by the compression technology or standard. Specifically, a profile may select a particular tool from all available tools for a video compression technology or standard as the only tool that can be used with that profile. Compliance also requires that the complexity of the coded video sequence be within the range defined at the level of the video compression technology or standard. In some cases, the maximum picture size, maximum frame rate, maximum reconstruction sample rate (eg, measured in megasamples per second), maximum reference picture size, etc. are limited by the level. The limits set by the levels may in some cases be further limited by virtual reference decoder (HRD) specifications and metadata for HRD buffer management signaled in the coded video sequence.

In one embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. Additional data may be included as part of the encoded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and/or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, and the like.

FIG. 6 shows a block diagram of a video encoder (603) according to an embodiment of the present disclosure. A video encoder (603) is included in the electronic device (620). Electronic device (620) includes a transmitter (640) (eg, transmitting circuitry). A video encoder (603) can be used instead of the video encoder (403) in the example of FIG.

A video encoder (603) receives video samples from a video source (60l) (not part of the electronic device (620) in the example of Figure 6) that may capture video images to be encoded by the video encoder (603). can do. In other examples, the video source (601) is part of the electronic device (620).

A video source (601) encodes a source video sequence encoded by a video encoder (603) in any suitable bit depth (e.g. 8 bits, 10 bits, 12 bits,...), in any color space ( e.g. BT.601 Y CrCB, RGB,...) and any suitable sampling structure (e.g. Y CrCb 4:2:0, Y CrCb 4:4:4) in the form of a digital video sample stream. can be provided. In a media provisioning system, the video source (601) may be a storage device that stores pre-prepared videos. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. Video data may be provided as multiple individual pictures that are imparted with motion when viewed in sequence. The picture itself may be organized as a spatial array of pixels, and each pixel may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following explanation will focus on samples.

According to one embodiment, the video encoder (603) encodes and compresses the pictures of the source video sequence into a coded video sequence (643) in real time or under any other time constraints required by the application. Can be done. Enforcing the appropriate coding speed is one of the functions of the controller (650). In some embodiments, controller (650) controls and is operably coupled to other functional units described below. Couplings are not shown for clarity. The parameters set by the controller (650) include rate control related parameters (picture skipping, quantization, lambda values for rate distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search. It can include ranges, etc. Controller (650) may be configured with other suitable functionality for video encoder (603) optimized for a particular system design.

In some embodiments, the video encoder (603) is configured to operate in a coding loop. As an oversimplified explanation, in one example, the coding loop includes a source coder (630) (e.g., responsible for creating symbols, such as a symbol stream based on input pictures and reference pictures to be coded), and a video encoder (630); It may include a (local) decoder (633) embedded in (603). The decoder (633) reconstructs the symbols to create sample data in a manner similar to that created by the (remote) decoder (any compression between the symbols and the coded video bitstream is disclosed). (This is because the video compression techniques considered in the subject matter are lossless). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since decoding of the symbol stream yields bit-accurate results regardless of the location of the decoder (local or remote), the contents of the reference picture memory (634) are also bit-accurate between the local and remote encoders. In other words, the prediction part of the encoder "sees" exactly the same sample values as reference picture samples that the decoder "sees" when using prediction during decoding. The basic principle of reference picture synchrony (and the drift that occurs when synchrony cannot be maintained, for example due to channel errors) is also used in several related fields.

The operation of the "local" decoder (633) may be similar to the operation of a "remote" decoder, such as the video decoder (510), already described in detail in connection with FIG. 5 in the preamble. However, referring also briefly to FIG. 5, since the symbols are available and the encoding/decoding of the symbols into the coded video sequence by the entropy coder (645) and parser (520) can be reversible, the buffer memory (515) ), and the entropy decoding portion of the video decoder (510), including the parser (520), may not be fully implemented in the local decoder (633).

As can be seen, any decoder technology other than parsing/entropy decoding that is present in a decoder must necessarily be present in substantially the same functional form in the corresponding encoder. Therefore, the disclosed subject matter focuses on decoder operation. A description of the encoder technology can be omitted since it is the reverse of the comprehensively described decoder technology. Only in certain areas more detailed explanation is required and is provided below.

During operation, in some examples, the source coder (630) predictively codes the input picture with reference to one or more previously coded pictures from the video sequence designated as "reference pictures." Motion compensated predictive coding may be performed to reduce the In this way, the coding engine (632) codes the differences between the pixel blocks of the input picture and the pixel blocks of the reference picture that may be selected as a prediction basis for the input picture.

A local video decoder (633) may decode coded video data of a picture that may be designated as a reference picture based on the symbols created in the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. When the coded video data may be decoded with a video decoder (not shown in FIG. 6), the reconstructed video sequence may be a replica of the source video sequence, usually with some errors. The local video decoder (633) may reproduce the decoding process that may be performed on the reference pictures by the video decoder and store the reconstructed reference pictures in the reference picture cache (634). In this way, the video encoder (603) locally stores a copy of the reconstructed reference picture that has common content (without transmission errors) with the reconstructed reference picture obtained by the far-end video decoder. obtain.

The predictor (635) may perform a predictive search of the coding engine (632). That is, for a new picture to be coded, the predictor (635) uses the sample data (as a candidate reference pixel block) or the reference picture's motion vector, block shape, etc. as a suitable prediction criterion for the new picture. The reference picture memory (634) may be searched for specific metadata that may function. The predictor (635) can operate on a sample block/pixel block basis to find suitable prediction criteria. In some cases, the input picture has prediction criteria derived from multiple reference pictures stored in a reference picture memory (634), as determined by the search results obtained in the predictor (635). Good too.

Controller (650) may manage coding operations of source coder (630), including, for example, setting parameters and subgroup parameters used to encode video data.

The outputs of all the aforementioned functional units are subjected to entropy coding in an entropy coder (645). The entropy coder (645) converts the symbols generated by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, and the like.

A transmitter (640) is created by an entropy coder (645) in preparation for transmission via a communication channel (660), which may be a hardware/software link to a storage device that stores encoded video data. The coded video sequence can be buffered. The transmitter (640) merges the coded video data from the video coder (603) with other data to be transmitted, such as coded audio data and/or an auxiliary data stream (source not shown). can do.

A controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to the respective picture. For example, pictures can often be assigned as any of the following picture types:

Intra pictures (I pictures) may be those that can be coded and decoded without using any other pictures in the sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures. Those skilled in the art are aware of the I-picture variants and their respective uses and characteristics.

A predicted picture (P-picture) may be one that can be coded and decoded by intra-prediction or inter-prediction using at most one motion vector and reference index to predict the sample values of each block.

Bidirectionally predicted pictures (B pictures) may be those that can be coded and decoded by intra-prediction or inter-prediction using up to two motion vectors and reference indices to predict the sample values of each block. Similarly, multi-predicted pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

A source picture is typically spatially subdivided into multiple sample blocks (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples, respectively) and may be coded on a block-by-block basis. A block may be predictively coded with reference to other (already coded) blocks determined by a coding assignment applied to each picture of the block. For example, blocks of an I-picture may be coded non-predictively or predictively with reference to previously coded blocks (spatial or intra-prediction) of the same picture. good. P-picture pixel blocks may be predictively coded via spatial prediction or via temporal prediction with reference to one pre-coded reference picture. A block of B pictures may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

The video encoder (603) is an ITU-T Rec. H. The coding operation may be performed according to a predefined video coding technique or standard, such as H.265. During operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancies in the input video sequence. Thus, the coded video data may conform to the syntax specified by the video coding technology or standard used.

In one embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the coded video sequence. Additional data may include temporal/spatial/SNR enhancement layers, redundant data in other forms such as redundant pictures or slices, SEI messages, VUI parameter set fragments, etc.

Video may be captured as multiple source pictures (video pictures) in chronological order. Intra-picture prediction (often abbreviated as "intra-prediction") exploits spatial correlations in a given picture, while inter-picture prediction exploits correlations (temporal or otherwise) between pictures. In one example, a particular picture being encoded/decoded, called the current picture, is divided into blocks. A block in the current picture can be coded by a vector called a motion vector if the block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video. . A motion vector points to a reference block of a reference picture and can have a third dimension that identifies the reference picture if multiple reference pictures are used.

In some embodiments, bi-prediction methods may be used in inter-picture prediction. According to the bi-prediction method, two reference pictures, such as a first reference picture and a second reference picture, which are each earlier in decoding order than the current picture in the video (but can be in the past and future, respectively, in the display order) Use reference pictures. The block in the current picture is coded by a first motion vector pointing to a first reference block in a first reference picture and a second motion vector pointing to a second reference block in a second reference picture. Can be done. A block can be predicted by a combination of a first reference block and a second reference block.

Additionally, merge mode techniques can be applied to inter-picture prediction to improve coding efficiency.

According to some embodiments of the present disclosure, predictions, such as inter-picture prediction and intra-picture prediction, are performed on a block-by-block basis. For example, according to the HEVC standard, pictures in a series of video pictures are divided into coding tree units (CTUs) for compression, and a CTU in a picture can be 64x64 pixels, 32x32 pixels, or 16x16 pixels. etc., have the same size. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree partitioned into one or more coding units (CUs). For example, a 64x64 pixel CTU can be divided into one 64x64 pixel CU, four 32x32 pixel CUs, or sixteen 16x16 pixel CUs. In one example, each CU is analyzed to determine the CU's prediction type, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and/or spatial predictability. Typically, each PU includes one luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding/decoding) are performed in units of prediction blocks. Using the luminance prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values (e.g., luminance values) such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc. .

FIG. 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. A video encoder (703) receives a processing block (e.g., a prediction block) of sample values in a current video picture in a series of video pictures and encodes the processing block into a coded picture that is part of a coded video sequence. It is configured as follows. In one example, a video encoder (703) is used in place of the video encoder (403) in the example of FIG.

In the HEVC example, the video encoder (703) receives a matrix of sample values for a processing block, such as a prediction block, such as an 8x8 sample. The video encoder (703) determines whether the processing block is best coded in intra mode, inter mode, or bi-predictive mode, for example using rate-distortion optimization. If the processing block is to be coded in intra mode, the video encoder (703) may encode the processing block into a coded picture using an intra prediction method. Additionally, if the processing block is to be coded in inter-prediction or bi-prediction mode, the video encoder (703) may encode the processing block into a coded picture using inter-prediction or bi-prediction methods, respectively. . In certain video coding techniques, the merge mode may be an inter-picture prediction submode that derives motion vectors from one or more motion vector predictors without the benefit of coded motion vector components outside the predictor. . In certain other video coding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (703) includes other components such as a mode determination module (not shown) for determining the mode of the processing block.

In the example of FIG. 7, the video encoder (703) includes an inter encoder (730), an intra encoder (722), a residual calculation unit (723), a switch (726), and a residual It includes an encoder (724), a general control unit (721), and an entropy encoder (725).

An inter-encoder (730) receives samples of a current block (e.g., a processing block) and compares the block to one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a later picture). , generate inter-prediction information (e.g. redundant information description by inter-encoding method, motion vector, merge mode information) and use any suitable technique based on the inter-prediction information to generate inter-prediction results (e.g. prediction blocks). is configured to calculate. In some examples, the reference picture is a decoded reference picture that is decoded based on encoded video information.

The intra-encoder (722) receives samples of the current block (e.g., a processing block), optionally compares the block with previously coded blocks in the same picture, and after transformation, calculates the quantized coefficients. and possibly also generate intra-prediction information (eg, intra-prediction direction information according to one or more intra-encoding methods). In one example, the intra encoder (722) also calculates intra prediction results (eg, predicted blocks) based on the intra prediction information and reference blocks in the same picture.

The overall control unit (721) is configured to determine overall control data and control other components of the video encoder (703) based on the overall control data. In one example, the overall control unit (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, when the mode is intra mode, the overall control unit (721) controls the switch (726) to select the intra mode result for use by the residual calculation unit (723), and also controls the intra prediction information. The entropy encoder (725) is controlled to select and include intra prediction information in the bitstream. Further, when the mode is inter mode, the overall control unit (721) controls the switch (726) to select the inter prediction result for use by the residual calculation unit (723), and also controls the inter prediction information The entropy encoder (725) is controlled to select and include inter prediction information in the bitstream.

The residual calculation unit (723) is configured to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (722) or the inter encoder (730). The residual encoder (724) operates on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to transform the residual data from the spatial domain to the frequency domain and generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). A residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be suitably used by the intra-encoder (722) and the inter-encoder (730). For example, the inter encoder (730) can generate decoded blocks based on decoded residual data and inter prediction information, and the intra encoder (722) can generate decoded blocks based on decoded residual data and intra prediction information. Blocks can be generated. In some examples, the decode block is suitably processed to generate a decode picture, which can be buffered in memory circuitry (not shown) and used as a reference picture.

The entropy encoder (725) is configured to format the bitstream to generate encoded blocks. The entropy encoder (725) is configured to include various information in the bitstream according to a suitable standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include integrated control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other suitable information in the bitstream. be done. Note that according to the disclosed subject matter, there is no residual information when coding a block in the merge submode of inter mode or bi-predictive mode.

FIG. 8 shows a diagram of a video decoder (810) according to another embodiment of the present disclosure. A video decoder (810) is configured to receive coded pictures that are part of a coded video sequence, decode the coded pictures, and generate reconstructed pictures. In one example, a video decoder (810) is used in place of the video decoder (410) in the example of FIG.

In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), coupled to each other as shown in FIG. and an intra decoder (872).

The entropy decoder (871) may be configured to reconstruct from the coded picture certain symbols representing syntactic elements that make up the coded picture. Such symbols are e.g. determined by the mode in which the block is coded (e.g., intra mode, inter mode, bi-predictive mode, later two merge submodes or other submodes), the intra decoder (872) or the inter decoder (872), respectively. Prediction information (e.g., intra-prediction information or inter-prediction information) that can identify the particular samples or metadata used for prediction by the decoder (880), such as residual information in the form of quantized transform coefficients. can be included. In one example, if the prediction mode is inter or bi-prediction mode, inter prediction information is provided to the inter decoder (880). Moreover, if the prediction type is an intra prediction type, intra prediction information is provided to the intra decoder (872). The residual information may be subjected to dequantization and provided to a residual decoder (873).

The inter-decoder (880) is configured to receive inter-prediction information and generate inter-prediction results based on the inter-prediction information.

The intra decoder (872) is configured to receive intra prediction information and generate prediction results based on the intra prediction information.

The residual decoder (873) performs inverse quantization, extracts inverse quantized transform coefficients, and processes the inverse quantized transform coefficients to transform the residual from the frequency domain to the spatial domain. It is composed of The residual decoder (873) may also require certain control information (such as including the quantizer parameter (QP)), and this information may be provided by the entropy decoder (871) ( The data path is not shown as it may only be low volume control information).

A reconstruction module (874) combines the residual output by the residual decoder (873) and the prediction result (possibly output by an inter or intra prediction module) in the spatial domain to generate a reconstructed video. is configured to form a reconstruction block that may be part of a reconstructed picture that may be part of a reconstructed picture. Note that other suitable operations, such as deblocking operations, may be performed to improve the visual quality.

It should be noted that video encoders (403), (603) and (703) and video decoders (410), (510) and (810) may be implemented using any suitable technique. In one embodiment, video encoders (403), (603) and (703) and video decoders (410), (510) and (810) may be implemented using one or more integrated circuits. . In other embodiments, video encoders (403), (603) and ( 703 ) and video decoders (410), (510) and (810) are implemented using one or more processors executing software instructions. can be done.

An Adaptive Loop Filter (ALF) with block-based filter adaptation can be applied by the encoder/decoder to reduce artifacts. For the luma component, one of multiple filters (e.g., 25 filters) can be selected for a 4x4 luma block, e.g., based on local gradient direction and activity. .

The ALF can have any suitable shape and size. Referring to FIG. 9, ALF(910) to (911) have a rhombic shape, such as a 5x5 rhombus for ALF(910) and a 7x7 rhombus for ALF(911). There is. In ALF (910), elements (920)-(932) may be used in a filtering process to form a diamond shape. Seven values (eg, C0 to C6) are available for elements (920) to (932). In ALF (911), elements (940)-(964) may be used in the filtering process to form a diamond shape. Thirteen values (eg, C0 to C12) are available for elements (940) to (964).

Referring to FIG. 9, in some examples two ALFs (910)-(911) having a diamond shape are used. A 5×5 diamond-shaped filter (910) can be applied to the chroma component (e.g., chroma block, chroma CB), and a 7×7 diamond-shaped filter (911) can be applied to the luma component (e.g., luma block, luma CB). CB). Other suitable shapes and sizes of ALFs can be used. For example, a 9x9 diamond shaped filter can be used.

The filter coefficients at the positions indicated by the values (eg, C0-C6 of (910) or C0-C12 of (920)) may be non-zero. Furthermore, if the ALF includes a clipping function, the clipping value at that location may be non-zero.

に基づいて、式（１）を用いて分類インデックスＣを導出することができる。 For block classification of luma components, a 4x4 block (or luma block, luma CB) can be categorized or classified into one of multiple (eg, 25) classes. In addition, the directional parameter D and the quantized value of the activity value A

Based on , the classification index C can be derived using equation (1).

を算出するために、１次元ラプラシアンを用いて、垂直方向、水平方向、２つの対角方向（例えば、ｄ１、ｄ２）の勾配ｇ_ｖ、ｇ_ｈ、ｇ_ｄ１、よびｇ_ｄ２をそれぞれ以下のように算出することができる。 Directionality parameter D and quantization value

To calculate the gradients g _v , g _h , g d1 , and g d2 in the vertical direction, horizontal _direction , and two diagonal directions (for example, d1, _d2 ) using the one-dimensional Laplacian, respectively, as follows. It can be calculated as follows.

Here, indices i and j refer to the coordinates of the top left sample within the 4x4 block, and R(k,l) indicates the reconstructed sample at coordinates (k,l). The directions (eg, d1 and d2) can refer to two diagonal directions.

To reduce the complexity of the block classification described above, a subsampled one-dimensional Laplacian computation can be applied. 10A-10D show the gradients g _v , g _h , g _d1 , and g Examples of sub-sampling positions used to calculate _d2 are shown. The same sub-sampling location can be used for gradient calculations in different directions. In FIG. 10A, the label "V" indicates the sub-sampling position for calculating the vertical gradient _gv . In FIG. 10B, the label "H" indicates the sub-sampling position for calculating the horizontal gradient g _h . In FIG. 10AC, the label "D1" indicates the sub-sampling position for calculating the d1 diagonal gradient g _d1 . In FIG. 10D, the label "D2" indicates the sub-sampling position for calculating the d2 diagonal gradient g _{d 2} .

と最小値

は、以下のように設定されることができる。 Maximum values of horizontal and vertical gradients g _v , g _h

and the minimum value

can be set as follows.

と最小値

は、以下のように設定されることができる。 Maximum value of two diagonal gradients g _d1 and g _d2

and the minimum value

can be set as follows.

The directional parameter D can be derived as follows based on the above values and the two thresholds t ₁ and t ₂ .

および

が真の場合、Ｄは０に設定される。 In step 1,

and

is true, D is set to 0.

の場合、ステップ３に進み、そうでない場合、ステップ４に進む。 In step 2,

If so, proceed to step 3, otherwise proceed to step 4.

の場合、Ｄは２に設定され、そうでない場合、Ｄは１に設定される。 In step 3,

, then D is set to 2; otherwise, D is set to 1.

の場合、Ｄは４に設定され、そうでない場合、Ｄは３に設定される。 In step 4,

, then D is set to 4; otherwise, D is set to 3.

The activity value A can be calculated as follows.

で表される。 A can be further quantized to a range of 0 to 4, and the quantization value is

It is expressed as

For the chroma components of a picture, no block classification is applied, so a single set of ALF coefficients can be applied for each chroma component.

Geometric transformations can be applied to the filter coefficients and corresponding filter clipping values (also referred to as clipping values). _Before filtering a block ( _e.g. a _4x4 luma block), _e.g. , rotation or diagonal and vertical flips can be applied to the filter coefficients f(k,l) and the corresponding filter clipping values c(k,l). The geometric transformation applied to the filter coefficients f(k,l) and the corresponding filter clipping values c(k,l) is equivalent to applying a geometric transformation to the samples in the region supported by the filter. could be. Geometric transformation can make different blocks to which ALF is applied more similar by aligning their orientations.

Three geometric transformations, including diagonal flip, vertical flip, and rotation, can be performed as described in equations (9)-(11), respectively.

Here, K is the size of the ALF or filter, 0≦k, and l≦K−1 are the coordinates of the coefficients. For example, position (0,0) is in the upper left corner of filter f or clipping value matrix (or clipping matrix) c, and position (K-1,K-1) is in the upper left corner of filter f or clipping value matrix (or clipping matrix ) in the lower right corner of c. Depending on the gradient values calculated for the block, a transformation can be applied to the filter coefficients f(k,l) and the clipping values c(k,l). An example of the relationship between the transformation and the four slopes is summarized in Table 1.

In some embodiments, the ALF filter parameters are signaled in a picture's adaptive parameter set (APS). In the APS, one or more sets (eg, up to 25 sets) of luma filter coefficients and clipping value indices may be signaled. In one example, one of the one or more sets may include luma filter coefficients and one or more clipping value indices. One or more sets (eg, up to eight sets) of chroma filter coefficients and clipping value indices may be signaled. To reduce signaling overhead, filter coefficients of different classifications (eg, with different classification indices) of the luma component can be merged. In the slice header, the index of the APS used for the current slice may be signaled.

In one embodiment, a clipping value index (also referred to as a clipping index) can be decoded from the APS. The clipping value index can be used to determine the corresponding clipping value, for example, based on the relationship between the clipping value index and the corresponding clipping value. This relationship can be predefined and stored in the decoder. In one example, this relationship includes a luma table of clipping value indexes and corresponding clipping values (e.g. used for luma CB), a chroma table of clipping value indexes and corresponding clipping values (e.g. used for chroma CB) It is described by a table such as The clipping value may depend on the bit depth B. The bit depth B may refer to the internal bit depth, the bit depth of the reconstructed samples within the CB to be filtered, etc. In some examples, tables (eg, luma tables, chroma tables) are obtained using equation (12).

Here, AlfClip is the clipping value, B is the bit depth (e.g., bitDepth), N (e.g., N=4) is the number of allowed clipping values, and (n-1) is the clipping value index (also known as clipping index or clipIdx). called). Table 2 is an example of a table obtained using equation (12) when N=4. The clipping index (n-1) can be 0, 1, 2, and 3 in Table 2, and n can be 1, 2, 3, and 4, respectively. Table 2 can be used for luma blocks or chroma blocks.

In the slice header of the current slice, one or more APS indices (eg, up to seven APS indices) may be signaled to specify the luma filter set that can be used for the current slice. The filtering process may be controlled at one or more suitable levels, such as picture level, slice level, CTB level, etc. In one embodiment, the filtering process can be further controlled at the CTB level. A flag may be signaled to indicate whether ALF applies to luma CTB. The luma CTB can select a filter set from among multiple fixed filter sets (eg, 16 fixed filter sets) and APS-signaled filter sets (also referred to as signaled filter sets). A filter set index may be signaled to the luma CTB to indicate the filter set to be applied (eg, a filter set of a plurality of fixed filter sets and signaled filter sets). Multiple fixed filter sets can be predefined and hard-coded in the encoder and decoder and can be referred to as predefined filter sets.

For chroma components, an APS index can be signaled in the slice header to indicate the chroma filter set used for the current slice. At the CTB level, if there is more than one chroma filter set in the APS, a filter set index may be signaled for each chroma CTB.

The filter coefficients can be quantized with a norm equal to 128. To reduce the multiplication complexity, bitstream conformance may be applied such that the coefficient values at non-center positions range from −2 ⁷ to 2 ⁷ −1 . In one example, the center position factor is not signaled in the bitstream and can be considered equal to 128.

In some embodiments, the syntax and semantics of clipping index and clipping value are defined as follows.

alf_luma_clip_idx[sfIdx][j] can be used to specify the clipping index of the clipping value to use before multiplying by the jth coefficient of the signaled luma filter denoted by sfIdx. Bitstream conformance requirements may include that the value of alf_luma_clip_idx[sfIdx][j] (sfIdx=0 to alf_luma_num_filters_signaled_minus1 and j=0 to 11) should be in the range from 0 to 3.

Luma filter clipping value AlfClipL[adaptation_parameter r_set_id], bitDepth is set equal to BitDepthY and clipIdx is set equal to alf_luma_clip_idx[ In response to being set equal to alf_luma_coeff_delta_idx[filtIdx][j], it can be derived as specified in Table 2.

alf_chroma_clip_idx[altIdx][j] can be used to specify the clipping index of the clipping value to use before multiplying by the jth coefficient of the alternative chroma filter with index altIdx. Bitstream conformance requirements may include that the value of alf_chroma_clip_idx[altIdx][j] (altIdx=0 to alf_chroma_num_alt_filters_minus1, j=0 to 5) should be in the range from 0 to 3.

chroma filter clipping value AlfClipC[adap tation_parameter_set_id][altIdx] has bitDepth set equal to BitDepthC and clipIdx set equal to alf_chroma_clip_idx Depending on being set equal to [altIdx][j], it can be derived as specified in Table 2.

In one embodiment, the filtering process can be described as follows. On the decoder side, if ALF is enabled for the CTB, the samples R(i,j) in the CU (or CB) can be filtered, so that the results are shown below using equation (13). A filtered sample value R'(i,j) is obtained as follows. In one example, each sample of the CU is filtered.

Here, f(k,l) indicates the decoded filter coefficient, K(x,y) is the clipping function, and c(k,l) indicates the decoded clipping parameter (or clipping value). The variables k and l can vary between -L/2 and L/2, where L indicates the filter length. Clipping function K(x,y)=min(y,max(-y,x)) corresponds to clipping function Clip3(-y,y,x). By incorporating the clipping function K(x,y), the loop filtering method (eg ALF) becomes a nonlinear process and can be called nonlinear ALF.

For non-linear ALF, multiple sets of clipping values can be provided in Table 3. In one example, the luma set includes four clipping values {1024, 181, 32, 6}, and the chroma set includes four clipping values {1024, 161, 25, 4}. The four clipping values of the luma set can be selected by approximately evenly dividing the entire range (eg, 1024) of the luma block's sample values (encoded with 10 bits) in the logarithmic domain. For chroma sets, the range can be 4 to 1024.

The selected clipping value can be encoded in the "alf_data" syntax element as follows: A clipping index corresponding to the selected clipping value may be encoded. The encoding scheme may be the same encoding scheme used to encode the filter set index.

In one embodiment, a virtual boundary filtering process may be used to reduce the ALF's line buffer requirements. Accordingly, modified block classification and filtering may be employed for samples near CTU boundaries (eg, horizontal CTU boundaries). The virtual boundary (1130) can be defined as a line by shifting the horizontal CTU boundary (1120) by “N _samples ” samples, where N _samples is a positive integer, as shown in FIG. 11A. It can be. In one example, N _samples is equal to 4 for the luma component and N _samples is equal to 2 for the chroma component.

Referring to FIG. 11A, a modified block classification may be applied to the luma component. In one example, only samples above the virtual boundary (1130) are used for the one-dimensional Laplacian gradient computation of the 4x4 block (1110) that is above the virtual boundary (1130). Similarly, referring to FIG. 11B, for the one-dimensional Laplacian gradient computation of the 4x4 block (1111) below the virtual boundary (1131), shifted from the CTU boundary (1121), the virtual boundary (1131) Only the lower samples are used. The quantization of the activity value A can be scaled appropriately by taking into account the reduction in the number of samples used for the one-dimensional Laplacian gradient calculation.

For the filtering process, symmetric padding operations at virtual boundaries can be used for both luma and chroma components. 12A-12F illustrate examples of such modified ALF filtering of luma components at virtual boundaries. If the sample to be filtered is located below the virtual boundary, adjacent samples located above the virtual boundary may be padded. If the sample to be filtered is located above the virtual boundary, adjacent samples located below the virtual boundary may be padded. Referring to FIG. 12A, adjacent sample C0 may be padded with sample C2 located below the virtual boundary (1210). Referring to FIG. 12B, adjacent sample C0 may be padded with sample C2 located above the virtual boundary (1220). Referring to FIG. 12C, adjacent samples C1-C3 may be padded with samples C5-C7 located below the virtual boundary (1230), respectively. Referring to FIG. 12D, adjacent samples C1-C3 may be padded with samples C5-C7 located above the virtual boundary (1240), respectively. Referring to FIG. 12E, adjacent samples C4-C8 may be padded with samples C10, C11, C12, C11, and C10, respectively, located below the virtual boundary (1250). Referring to FIG. 12F, adjacent samples C4-C8 may be padded with samples C10, C11, C12, C11, and C10, respectively, located above the virtual boundary (1260).

In some examples, the above description may be suitably adapted when the sample and neighboring samples are located to the left (or right) and to the right (or left) of the virtual boundary.

Equation (12) applied to the luma component or chroma component can be rewritten as equation (12').

As can be seen from equation (12'), the clipping value (eg, AlfClip) in ALF depends on the bit depth B. Furthermore, the clipping value depends on 2 ^B 2 ^(B(1-n)/N) . In one embodiment, if N is specified (eg, N=4), 2 ^(B(1-n)/N) depends on the bit depth B and the clipping index (n-1). Therefore, 2 ^B 2 ^(B(1-n)/N) before rounding is not proportional to 2 ^B , and determining the clipping value using equation (12) or (12') That embodiment may not be efficient.

Aspects of the present disclosure include deriving clipping values used in filtering (eg, nonlinear loop filtering). A clipping value (eg, AlfClip) used in ALF may be determined based on the bit depth B and the clipping index. According to aspects of the present disclosure, the dependence of clipping values on bit depth B and clipping index can be separated to improve coding efficiency.

A clipping value can be associated with a clipping index. The clipping value can be based on a multiplication of the first function and the second function. In one embodiment, the clipping value may be determined by a composite function that is a multiplication of a first function and a second function. The first function can depend on the bit depth B, which is independent of the clipping index, and therefore does not depend on the clipping index. In one example, the first function is proportional to 2 ^B , thus improving coding efficiency. The second function is independent of bit depth B because it depends on the clipping index and can be independent of bit depth B.

In one embodiment, equation (14) may be used to derive the clipping value (eg, AlfClip) used in the ALF of luma CB and chroma CB.

Here, AlfClip is the clipping value, B means the internal bit depth of the ALF or the bit depth of the input sample, the constant value α means the coefficient that controls the strength of the nonlinear ALF filter, and m is the clipping index. , N indicates the number of allowed clipping values (for example, 4). Clipping index m may be 0, . . . and N-1. The constant value α can include, but is not limited to, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, and 2.8.

In the example shown in equation (14), the first function is 2 ^B and the second function is 2 ^−α*m . The first function depends on the bit depth B and is independent of the clipping index m. Therefore, the first function does not depend on the clipping index m. The second function depends on the clipping index m and is independent of the bit depth B. Therefore, the second function is independent of bit depth B. The value 2 ^B-α*m can be rounded (eg, rounded up or down to an integer value) to generate a clipping value.

The same relationship as shown in equation (14) can be used for the luma component (or CB) and the chroma component (or CB). For luma CB, the constant value α may be the first constant value. For chroma CB, the constant value α may be a second constant value. In some examples, the first constant value is the same as the second constant value. In some examples, the first constant value is different than the second constant value.

The relationship between clipping value and clipping index can be specified using an equation such as equation (14) shown above. This relationship may also be specified using other suitable methods, such as look-up tables (eg, as shown in FIGS. 13-14). FIG. 13 is an exemplary table illustrating the relationship between clipping values and bit depth B and clipping index m, according to one embodiment of the present disclosure. The clipping values in the table can be determined based on equation (14) where the constant value α is 2.3 and N is 4. In one example, this table can be applied to a luma CB and the clipping value can be referred to as a luma clipping value (eg, AlfClip _L ).

Referring to FIG. 13, if the clipping index (eg, clipIdx) is 0, the clipping value determined using equation (14) is the same as the clipping value determined using equation (13). If the clipping index (e.g., clipIdx) is 1, 2, or 3, the clipping value determined using equation (14) (highlighted in italics) is determined using equation (13). clipping value (crossed out). For example, when bitDepth is 8 and clipIdx is 1, the clipping value determined using equation (14) is 52, and the clipping value determined using equation (13) is 64.

FIG. 14 is an example table illustrating the relationship between clipping values and bit depth B and clipping index m, according to one embodiment of the present disclosure. The clipping values in the table are determined based on equation (14) where the constant value α is 2.6 and N is 4. In one example, the table of FIG. 14 can be applied to chroma CB, and the clipping value can be referred to as a chroma clipping value (eg, AlfClip _C ).

Referring to FIG. 14, if the clipping index (eg, clipIdx) is 0, the clipping value determined using equation (14) is the same as the clipping value determined using equation (13). If the clipping index (e.g., clipIdx) is 1, 2, or 3, the clipping value determined using equation (14) (highlighted in italics) is determined using equation (13). clipping value (crossed out). For example, when bitDepth is 8 and clipIdx is 1, the clipping value determined using equation (14) is 42, and the clipping value determined using equation (13) is 64.

As mentioned above, the luma component (eg, luma CB) and chroma component (eg, chroma CB) can use different clipping values derived by equation (14). For example, when deriving the clipping value using equation (14), the constant value α is 2.3 in the case of the luma component shown in FIG. 13, and the constant value α is 2.3 in the case of the chroma component shown in FIG. It becomes 2.6. In some examples, a constant value α of 2.3 provides optimal coding efficiency for the luma component, and a constant value α of 2.6 provides optimal coding efficiency for the chroma component.

As mentioned above, the clipping index can be used to determine the corresponding clipping value, for example, based on the relationship between the clipping index and the corresponding clipping value. This relationship can be predefined and stored in the decoder. This relationship can be described by a table (eg, as shown in FIGS. 13-14), an equation, etc. that can be predefined and stored in the decoder. In some examples, parameters used in the table and/or equations (eg, bit depth B, constant values α, N) are signaled.

Determining the ALF clipping value using equation (14) may be different than using equation (12). Some of the differences are illustrated below (highlighted in italics) and in the tables shown in Figures 13-14, where strikethrough text indicates deleted text.

alf_luma_clip_idx[sfIdx][j] may specify the clipping index of the corresponding clipping value to use before multiplying by the jth coefficient of the signaled luma filter indicated by sfIdx. In one example, a requirement for bitstream compatibility is that the value of alf_luma_clip_idx[sfIdx][j] (sfIdx=0..alf_luma_num_filters_signaled_minus1 and j=0..11) is within the range of 0 to 3.

Luma filter clipping value with element _{AlfClip L} [adaptation_parameter_set_id] [filtIdx] [j] (filtIdx=0..NumAlfFilters-1 and j=0..11) ramet_set_id], for example, if bitDepth is equal to BitDepth _Y and clipIdx is set equal to alf_luma_clip_idx[alf_luma_coeff_delta_idx[filtIdx]][j], which can be derived as specified in the table shown in FIG.

alf_chroma_num_alt_filters_minus1+1 can specify the number of alternative filters for the chroma component.

alf_chroma_clip_flag[altIdx] equal to 0 can specify to apply linear adaptive loop filtering to the chroma component when using the chroma filter with index altIdx, and alf_chroma_clip_flag[altIdx] equal to 1 can specify that linear adaptive loop filtering is applied to the chroma component when using the chroma filter with index altIdx; When using a filter, you can specify that nonlinear adaptive loop filtering be applied to the chroma components. If not present, alf_chroma_clip_flag[altIdx] can be inferred to be equal to 0.

alf_chroma_coeff_abs[altIdx][j] can specify the absolute value of the j-th chroma filter coefficient of the alternative chroma filter with index altIdx. If alf_chroma_coeff_abs[altIdx][j] is not present, it can be inferred to be equal to 0. In one example, a requirement for bitstream compatibility is that the value of alf_chroma_coeff_abs[altIdx][j] is within the range of 0 to 2 ⁷ −1.

The order k of the exponential Golomb binarization uek(v) can be set equal to three.

alf_chroma_coeff_sign[altIdx][j] can specify the sign of the j-th chroma filter coefficient of the alternative chroma filter with index altIdx as follows.
If alf_chroma_coeff_sign[altIdx][j] is equal to 0, the corresponding chroma filter coefficient can have a positive value.
Otherwise (if alf_chroma_coeff_sign[altIdx][j] is equal to 1), the corresponding chroma filter coefficient can have a negative value.

If alf_chroma_coeff_sign[altIdx][j] is not present, it can be inferred to be equal to 0.

Chroma filter coefficient _{AlfCoeff C} with element AlfCoeff C [adaptation_parameter_set_id] [altIdx] [j] (altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5) [adaptation_parameter_set_id][altIdx] shall be derived as follows: Can be done.

AlfCoeff _C [adaptation_parameter_set_id][altIdx][j]=alf_chroma_coeff_abs[altIdx][j]*(1-2*alf_chroma_coeff_sign[altIdx][ j]) Formula (15)

In one example, the value of AlfCoeff _C [adaptation_parameter_set_id] [altIdx] [j] (altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5) is -2 ⁷ -1 or more 2 ⁷ Must be within the range of -1 or less is a requirement for bitstream conformance.

alf_chroma_clip_idx[altIdx][j] can specify the clipping index of the clipping value to be used before multiplying by the j-th coefficient of the alternative chroma filter with index altIdx. In one example, a requirement for bitstream compatibility is that the value of alf_chroma_clip_idx[altIdx][j] (altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5) is within the range of 0 to 3.

Clipping value AlfClip of the chroma filter with element AlfClip _C [adaptation_parameter_set_id] [altIdx] [j] (altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5) _C [adaptation_parameter_set_id] [altIdx], for example, if bitDepth is BitDepth _C and clipIdx is set equal to alf_chroma_clip_idx[altIdx][j], which can be derived as specified in the table shown in FIG.

FIG. 15 is a flowchart outlining a process (1500) according to one embodiment of the present disclosure. Process (1500) may be used to reconstruct CBs in pictures of a coded video sequence. In various embodiments, the process (1500) includes processing circuitry in the terminal devices (310), (320), (330) and (340), processing circuitry that performs the functions of a video encoder (403), a video decoder (410). ), a processing circuit that performs the function of a video decoder (510), a processing circuit that performs the function of the video encoder (603), etc. In some embodiments, the process (1500) is implemented with software instructions, such that when the processing circuitry executes the software instructions, the processing circuitry executes the process (1500). This process starts at (S1501) and proceeds to (S1510).

In (S1510), coding information of CBs in pictures of the coded video sequence may be decoded. The encoded information may indicate a clipping index m. The clipping index m can indicate the clipping value of a filter applied to the CB. The reconstructed samples within the CB may be filtered by a filter. In some examples, the filter is a nonlinear ALF corresponding to equation (13). The filter may be characterized by a filter coefficient (e.g., f(k,l)) and a corresponding clipping value (e.g., c(k,l)), as described above with reference to equation (13). Can be done. For example, the filter includes a clipping function (eg, Clip3(-y, y, x)) that is dependent on the clipping value (eg, as shown in equation (13)). The CB can be a luma CB or a chroma CB.

The filtering process may be controlled at one or more suitable levels, such as picture level, slice level, CTB level, etc. In some examples, coded information indicating the clipping index is signaled at the picture level in the APS of the picture. The coded information signaled in the APS may include at least one signaled filter set including filter coefficients and clipping indices. Also, a filter set index can be signaled to the CTB, for example to indicate which ALF to use for the CB. The filter set index can be received by a decoder, for example. Based on the filter set index, a filter can be determined from multiple filter sets and then applied to the CB. In some embodiments, multiple filter sets may be formed by at least one signaled filter set and multiple predefined filter sets.

At (S1520), a clipping value (eg, AlfClip, AlfClipL, or AlfClipC) associated with the clipping index may be determined. The clipping value can be based on a multiplication of the first function and the second function. The first function may depend on the bit depth B and may be independent of the clipping index m, and the second function may depend on the clipping index m and be independent of the bit depth B. be able to. The first function can be proportional to 2 ^B , thus improving coding efficiency. In one embodiment, the clipping values are integers, the first function is 2 ^B and the second function is 2 ^{- αm} , where α is a constant value associated with the strength of the filter. In one example, the clipping value (eg, an integer) is determined based on 2 ^B 2 ^−αm .

As mentioned above, in one example, the clipping value is equal to rounding the value of 2 ^B 2 ^-αm to an integer. The value 2 ^B 2 ^−αm can be rounded up or down to an integer (eg, to the nearest integer) to generate a clipping value. In one example, the clipping value is 2 ^B 2 ^-αm rounded to the nearest integer.

The bit depth B may be the internal bit depth, the bit depth of the reconstructed samples within the CB, etc. Additionally, the clipping index m may be a non-negative integer. In one example, the clipping index is one of a range from 0 to the number of allowed clipping values of the clipping index N (eg, 4). The clipping index is 0, . ．． ．． , (N-1). In one example, N is 4, so the clipping index is one of 0, 1, 2, and 3. The bit depth B and the number N of allowed clipping values can be predefined and stored in the encoder and/or decoder. Alternatively, bit depth B and/or N can be signaled to the decoder.

The constant value α may be a first constant value based on the fact that the CB is a luma CB, and the constant value α may be a second constant value based on the fact that the CB is a chroma CB. can. The first constant value and the second constant value may be the same or different. Optimal values can be used for the first constant value and the second constant value to improve coding efficiency. In one example, the first constant value is 2.3 for luma CB and the second constant value is 2.6 for chroma CB. The first constant value and/or the second constant value may be predefined and stored in the encoder and/or decoder. Alternatively, the first constant value and/or the second constant value can be signaled to the decoder.

In one embodiment, the clipping value can be determined from the clipping index based on a relationship between the clipping value and the clipping index. This relationship can indicate that the clipping value is determined by the multiplication of a first function (eg, proportional to 2 ^B ) and a second function. In one example, a lookup table (e.g., the table for luma CB in FIG. 13, the table for chroma CB in FIG. 14) may be used to determine the clipping value based on clipping index m and bit depth B. can.

In (S1530), a filtered CB can be generated by applying a filter to the CB.

Process (1500) can be adapted as appropriate. Steps of process (1500) may be modified and/or omitted. Also, additional steps can be added. Any suitable implementation order may be used.

The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, FIG. 16 depicts a computer system (1600) suitable for implementing certain embodiments of the disclosed subject matter.

Computer software may be assembled, compiled, linked, or subjected to such mechanisms to be directly or interpretably executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc. can be encoded using any suitable machine code or computer language to create code that includes instructions that can be executed by, etc.

The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

The example components shown in FIG. 16 for computer system (1600) are exemplary in nature and do not suggest any limitations as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure. That's not my intention either. The configuration of components should not be construed as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system (1600).

Computer system (1600) may include certain human interface input devices. Such human interface input devices include, for example, tactile input (keystrokes, swipes, data glove movements, etc.), audio input (voice, clap, etc.), visual input (gestures, etc.), olfactory input (not shown) can respond to input by one or more users. Human interface devices include audio (voices, music, environmental sounds, etc.), images (scanned images, photographic images obtained from still image cameras, etc.), video (2D video, 3D video including stereoscopic video, etc.), etc. It can also be used to capture specific media that is not necessarily directly related to conscious human input.

Input human interface devices include a keyboard (1601), mouse (1602), trackpad (1603), touch screen (1610), data glove (not shown), joystick (1605), microphone (1606), and scanner (1607). ), cameras (1608) (only one of each shown).

Computer system (1600) may also include certain human interface output devices. Such human interface output devices may stimulate one or more user's senses through, for example, tactile output, sound, light, and smell/taste. Such a human interface output device may include a tactile output device (e.g., a touch screen (1610), a data glove (not shown), or a tactile feedback device that has tactile feedback through a joystick (1605) but does not function as an input device). ), audio output devices (speakers (1609), headphones (not shown), etc.), visual output devices (screens (1610) including CRT screens, LCD screens, plasma screens, OLED screens (each including a touch screen) with or without input capability, with or without haptic feedback capability, respectively; some of them output two-dimensional visual output or output in three or more dimensions through means such as stereographic output; virtual reality glasses (not shown), holographic displays and smoke tanks (not shown), and printers (not shown).

The computer system (1600) includes human accessible storage devices and their associated media, e.g. optical media including CD/DVD ROM/RW (1620) with media (1621) such as CD/DVD, thumb drive, etc. (1622), removable hard drives or solid state drives (1623), traditional magnetic media such as tapes and floppy disks (not shown), and specialized ROM/ASIC/PLD-based devices such as security dongles (not shown). It can also include things such as "no".

Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not encompass transmission media, carrier waves, or other transitory signals. .

Computer system (1600) can further include an interface (1654) to one or more communication networks (1655). The network may be wireless, wired, optical, for example. Networks can also be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial TV; Including vehicle use including CANBus, industrial use, etc. A particular network typically requires an external network interface adapter connected to a particular general purpose data port or peripheral bus (1649), such as a USB port on a computer system (1600). Others are typically integrated into the core of the computer system (1600) by connecting to the system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). ). Using any of these networks, computer system (1600) can communicate with other entities. Such communications may be unidirectional, receive only (e.g. broadcast TV), unidirectional transmit only (e.g. CANbus to certain CANbus devices), or bidirectional, e.g. other networks using local or wide area digital networks. It may be a transmission to a computer system. Specific protocols and protocol stacks can be used with each of these networks and network interfaces mentioned above.

The aforementioned human interface devices, human accessible storage, and network interfaces may be connected to the core (1640) of the computer system (1600).

The core (1640) includes dedicated programmable processing units in the form of one or more central processing units (CPUs) (1641), graphics processing units (GPUs) (1642), and field programmable gate areas (FPGAs) (1643). , hardware accelerators (1644) for specific tasks, graphics adapters (1650), and the like. These devices connect the system bus (1648) along with internal mass storage (1647) such as read-only memory (ROM) (1645), random access memory (1646), non-user accessible internal hard drives, and SSDs. It may be connected via. In some computer systems, the system bus (1648) is accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripherals can be connected directly to the core's system bus (1648) or via a peripheral bus (1649). In one example, a display (1610) can be connected to a graphics adapter (1650). Peripheral bus architectures include PCI, USB, and the like.

The CPU (1641), GPU (1642), FPGA (1643), and accelerator (1644) may combine to execute specific instructions that may constitute the aforementioned computer code. The computer code can be stored in ROM (1645) or RAM (1646). Transitional data may also be stored in RAM (1646), while persistent data may be stored, for example, in internal mass storage (1647). Any memory by using cache memory that can be closely associated with one or more CPUs (1641), GPUs (1642), mass storage (1647), ROM (1645), RAM (1646), etc. Allows for fast storage and retrieval on your device.

The computer-readable medium can include computer code for performing various computer-implemented operations. The media and computer code may be specially designed and constructed for the purposes of this disclosure, or they may be of the type well known and available to those skilled in the computer software arts. obtain.

By way of example and not limitation, a computer system (1600) having an architecture, and in particular a core (1640), may include a processor (CPU, GPU, FPGA) executing software embodied in one or more tangible computer-readable media. , accelerators, etc.). Such computer-readable media includes user-accessible mass storage as introduced above, and core internal mass storage (1647) or core memory of a non-transitory nature, such as ROM (1645). (1640) may be media associated with specific storage. Software implementing various embodiments of the present disclosure can be stored on such devices and executed by the core (1640). The computer-readable medium can include one or more memory devices or chips, depending on particular needs. The software defines the data structures stored in the RAM (1646) in the core (1640), specifically the processors therein (including CPUs, GPUs, FPGAs, etc.), and A particular process or a particular portion of a particular process described herein may be executed, including language that modifies such data structures in accordance with the process. In addition, or in the alternative, the computer system may include circuitry that can operate in place of or in conjunction with software to perform certain processes or certain portions of certain processes described herein, such as , accelerator (1644)) as a result of hard-wired or embedded logic. References to software can optionally include logic, and vice versa. Reference to a computer-readable medium includes, as appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both. Can be done. This disclosure includes any suitable combination of hardware and software.

Appendix A: Acronyms JEM: Joint Exploration Model
VVC: Versatile Video Coding
BMS: Benchmark Set
MV: Motion Vector
HEVC: High Efficiency Video Coding
MPM: most probable mode
WAIP: Wide-Angle Intra Prediction
SEI: Supplementary Enhancement Information
VUI: Video Usability Information
GOP: Groups of Pictures
TU: Transform Units
PU: Prediction Units
CTU: Coding Tree Units
CTB: Coding Tree Blocks
PB: Prediction Blocks
HRD: Hypothetical Reference Decoder
SDR: standard dynamic range
SNR: Signal Noise Ratio
CPU: Central Processing Units
GPU: Graphics Processing Units
CRT: Cathode Ray Tube
LCD: Liquid-Crystal Display
OLED: Organic Light-Emitting Diode
CD: Compact Disc
DVD: Digital Video Disc
ROM: Read-Only Memory
RAM: Random Access Memory
ASIC: Application-Specific Integrated Circuit
PLD: Programmable Logic Device
LAN: Local Area Network
GSM: Global System for Mobile Communications
LTE: Long-Term Evolution
CANBus: Controller Area Network Bus
USB: Universal Serial Bus
PCI: Peripheral Component Interconnect
FPGA: Field Programmable Gate Areas
SSD: Solid-State Drive
IC: Integrated Circuit
CU: Coding Unit
PDPC: Position Dependent Prediction Combination
ISP: Intra Sub-Partitions
SPS: Sequence Parameter Setting

Although this disclosure has described some exemplary embodiments, there are alterations, permutations, and various alternative equivalents that fall within the scope of this disclosure. Accordingly, those skilled in the art will be able to devise various systems and methods not expressly shown or described herein, but which embody the principles of, and are within the spirit and scope of, the present disclosure. will be understood.

1648 system bus
1650 graphics adapter
1654 network interface

Claims (8)

A method for video decoding in a decoder, the method comprising:
decoding coding information of a coding block in a picture of a coded video sequence, the coding information indicating a clipping index m, the clipping index m being a clipping value of a filter applied to the coding block; showing the steps and
determining the clipping value associated with the clipping index m, the clipping value being based on a multiplication of a first function and a second function, wherein the first function is based on a bit depth; B and independent of said clipping index m, said second function being dependent on said clipping index m and independent of said bit depth B, and said second function for each said clipping index m. 2 is independent of N, where N is the total number of allowed clipping values including the clipping value;
generating a filtered coding block by applying the filter corresponding to the clipping value to the coding block;
including;
the clipping value is an integer;
The step of determining the clipping value includes determining the clipping value based on 2 ^B 2 ^−αm , the first function is 2 ^B and the second function is 2 −αm ^. , α is a constant value associated with the strength of the filter,
The constant value α is a first constant value when the coding block is a luma coding block, and the constant value α is different from the first constant value when the coding block is a chroma coding block. The second constant value, the method for video decoding. 2. The method of claim 1, wherein the filter is a nonlinear adaptive loop filter that includes a clipping function that depends on the clipping value. The method of claim 1 , wherein the clipping index m is one of 0, 1, 2, and 3. 2. The method of claim 1 , wherein the first constant value is 2.3 and the second constant value is 2.6. 2. The method of claim 1, wherein the coding information indicating the clipping index m is signaled in an adaptive parameter set (APS) of the picture. receiving a filter set index for the coding block;
6. The method of claim 5 , further comprising determining the filter from a plurality of filter sets based on the filter set index. A device for video decoding,
a processing circuit configured to carry out the method according to any one of claims 1 to 6 ;
A device comprising: A program configured to cause a computer to execute the method according to any one of claims 1 to 6 .

Images