EP4717029A1

EP4717029A1 - Video compression with separated luma and chroma planes

Info

Publication number: EP4717029A1
Application number: EP24739870.4A
Authority: EP
Inventors: Xiang Li; Debargha Mukherjee; Yaowu Xu; Jingning Han
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2023-06-29
Filing date: 2024-06-11
Publication date: 2026-04-01
Also published as: CN121420535A; WO2025006167A1

Abstract

A compressed bitstream is received. The compressed bitstream includes, with respect to a group of coding tree units, a luma plane followed by a chroma plane. The luma plane constitutes data relating to luma pixel values and the chroma plane constitutes data relating to chroma pixel values. A luma block of the luma plane is decoded based on a luma reference block included in another luma plane. A chroma block of the chroma plane is decoded based on a reference block, wherein the chroma block is collocated with the luma block.

Description

VIDEO COMPRESSION WITH SEPARATED LUMA AND CHROMA PLANES

BACKGROUND

[0001] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high-definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including encoding or decoding techniques.

SUMMARY

[0002] One general aspect of the teachings herein is a method that includes receiving a compressed bitstream that includes, with respect to a group of coding tree units, a luma plane followed by a chroma plane, where the luma plane constitutes data relating to luma pixel values and the chroma plane constitutes data relating to chroma pixel values; decoding a luma block of the luma plane based on a luma reference block included in another luma plane; and decoding a chroma block of the chroma plane based on a reference block, where the chroma block is collocated with the luma block.

[0003] Another embodiment of this aspect is a device that includes a processor configured to perform the actions of the method. Another embodiment is a device that includes a memory and a processor that is configured to execute instructions stored in the memory to perform the actions of the method. Another embodiment is a non-transitory computer-readable storage medium that includes executable instructions that, when executed by a processor, facilitate performance of the operations of the method. Another embodiment is a non-transitory computer-readable storage medium having stored thereon an encoded bitstream that is configured for decoding by the operations of the method.

[0004] Implementations may include one or more of the following features.

[0005] The method where the reference block can be decoded from the luma plane.

[0006] The method can include decoding the reference block from the luma plane and obtaining a post-filtered reference block from the reference block, where decoding the chroma block of the chroma plane based on the reference block can include decoding the chroma block based on the post-filtered reference block.

[0007] The method where the reference block is decoded from the chroma plane.

[0008] The group of coding tree units can be a frame. The group of coding tree units can be a tile of a frame, where the frame is composed of multiple tiles. The group of coding tree units can be a segment of a frame, where the frame is composed of multiple segments. The group of coding tree units can be a slice of a frame, where the frame is composed of multiple slices.

[0009] The compressed bitstream can include a coding information plane with respect to the group of coding tree units. The coding information plane can be included in the luma plane. The chroma plane can be a first chroma plane, and the compressed bitstream can further include, with respect to a group of coding tree units (CTUs), a second chroma plane following the first chroma plane.

[0010] The method can include decoding, from the luma plane, a first tree corresponding to a first partitioning of blocks of the luma plane; and decoding, from the chroma plane, a second tree corresponding to a second partitioning of blocks of the chroma plane.

[0011] A first structure of the first tree can be different from a second structure of the second tree.

[0012] The first tree can be scanned using a first scanning order and the second tree can be scanned using a second scanning order that is different from the first scanning order. [0013] A first scanning order can be used to scan coding tree units of the luma plane and a second scanning order that is different from the first scanning order can be used to scan coding tree units of the chroma plane.

[0014] Decoding the chroma block of the chroma plane based on the reference block can include generating a list of merge candidates; and selecting the reference luma block based on index into the list of the merge candidates decoded from the compressed bitstream. The list of merge candidates can include temporal prediction candidates, where each of the temporal prediction candidates is based on a respective motion vector of a sub-block of the luma block of the luma plane.

[0015] The method can include obtaining an intermediate prediction block corresponding to one of the temporal prediction candidates; and applying a filter to the intermediate prediction block to obtain a final prediction block.

[0016] The method can include obtaining a first intermediate prediction block corresponding to one of the temporal prediction candidates; obtaining a second intermediate prediction block based on a cross-component prediction from the luma block; and combining the first intermediate prediction block and the second intermediate prediction block to obtain a final prediction block. The list of merge candidates can include at least one candidate that is based on a prediction of the chroma block obtained from the luma block. The list of merge candidates can include at least one candidate that is based on an intra block copy mode.

[0017] The method can include filtering the at least one candidate based on filter coefficients obtained via a cross-component filtering operation.

[0018] The list of merge candidates can include a prediction candidate obtained using an intra-prediction mode.

[0019] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying figures. [0020] It will be appreciated that aspects can be implemented in any convenient form. For example, aspects may be implemented by appropriate computer programs which may be carried on appropriate carrier media which may be tangible carrier media (e.g. disks) or intangible carrier media (e.g. communications signals). Aspects may also be implemented using suitable apparatus which may take the form of programmable computers running computer programs arranged to implement the methods and/or techniques disclosed herein. Aspects can be combined such that features described in the context of one aspect may be implemented in another aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The description herein makes reference to the accompanying drawings described below, wherein like reference numerals refer to like parts throughout the several views. [0022] FIG. 1 is a schematic of a video encoding and decoding system.

[0023] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station.

[0024] FIG. 3 is a diagram of a typical video stream to be encoded and subsequently decoded.

[0025] FIG. 4 is a block diagram of an encoder according to implementations of this disclosure.

[0026] FIG. 5 is a block diagram of a decoder according to implementations of this disclosure.

[0027] FIG. 6 is a block diagram of a representation of a portion of a frame according to implementations of this disclosure. [0028] FIG. 7 is a block diagram of an example of a quad-tree representation of a block according to implementations of this disclosure.

[0029] FIG. 8A is a block diagram of an example of recursive partitioning of a coding block into prediction blocks.

[0030] FIG. 8B is a block diagram of an example of extended partition types of a coding block according to implementations of this disclosure.

[0031]

[0032] FIG. 9 is a flowchart of a technique for decoding coding tree units structured into planes of color components.

[0033] FIG. 10 illustrates examples of compressed bitstreams structured into color component planes.

[0034] FIG. 11 illustrates an example of cross-component filtering.

[0035] FIG. 12 illustrates an example of a neighborhood of an intermediate pixel of an intermediate prediction block.

[0036] FIG. 13 illustrates an example of obtaining a cross-component prediction.

[0037] FIG. 14 illustrates an example of obtaining a cross-component prediction.

DETAILED DESCRIPTION

[0038] Video compression schemes may include breaking respective images, or frames, of a video stream into smaller portions, such as blocks, or coding tree units (CTUs), and generating an encoded bitstream using techniques to limit the information included for respective CTUs thereof. The bitstream can be decoded to re-create the source frames from the limited information. Encoding CTUs to or decoding CTUs from a bitstream can include predicting the values of pixels or CTUs based on similarities with other pixels or CTUs in the same frame which have already been coded. Those similarities can be determined using intra prediction, which attempts to predict the pixel values of a coding unit (CU) of a CTU using pixels peripheral to the CU (e.g., pixels that are in the same frame as the CU, but which are outside the CU).

[0039] A CU includes a luminance, also referred to as luma, component and two chrominance, also referred to as chroma, components. These luma and chroma components may in some case be referred to as a luma block and chroma blocks. The luma component of a CU may, for example, be expressed within a Y plane of the CU and the chroma components may be expressed either within U and V planes or Cr and Cb planes of the CU. The luma component is understood to include some number of luma samples and each chroma component is understood to include some number of chroma samples. Generally, the luma samples provide measures of brightness throughout a subject CU and thus represents the structural qualities of the video content of the subject CU, whereas the chroma samples provide measures of color throughout the subject CU. Because of this, conventional video compression schemes often use finer prediction approaches for predicting luma components of CUs than chroma components thereof. Such schemes may also use approaches directed to predicting those chroma components from the predicted luma components.

[0040] A frame can be divided into blocks (CTUs) of the maximum possible size, often also referred to as superblocks or macroblocks. The partitioning strategies of CTUs into CUs are signaled in a compressed bitstream. The partitioning of a CTU can be hierarchical. That is, a CTU may be recursively divided into progressively smaller CUs. The partitioning of a CTU is communicated in the form of, or referred to as, a partition. The partition tree is a hierarchical structure that represents the division of a CTU into smaller coding blocks. Each node in the tree corresponds to a block, with the root node representing the CTU and the leaf nodes representing the smallest CUs.

[0041] At least in the case of a CU that is inter-predicted, conventional codecs signal a single block partition tree for the CU. The block partition tree is used for both the luma and chroma components of the CU. Thus, the same partition structure is applied to each of the luma Y, chroma U, and chroma V components of the CU. Inter-prediction broadly refers to any coding mode that leverages temporal similarities between the block in a current frame and other blocks within another frame that is different from the current frame.

[0042] To illustrate, in H.265/HEVC (ITU-T Rec. H.265, “High Efficiency Video Coding”, December 2016), the coding tree of a coding tree unit (CTU) is shared by Y, U, and V components. In the joint exploration model (JEM), versatile video coding (VVC) codec, a single tree structure is used for P and B slices. However, the luma and chroma components may be encoded separately in I slices. As such, a luma CTU (containing only one luma coding tree block of the original CTU) forms one coding tree, and a chroma CTU (containing only two chroma coding tree blocks of the original CTU) forms a chroma separate tree (CST). The CST design in I slices may also be referred as “CTU dual tree.”

[0043] Another conventional approach interleaves luma component partition trees (i.e., luma trees) with chroma component partition trees (i.e., chroma trees). To illustrate, a bitstream may include, in the following order and with additional data (e.g., residual data) interspersed therebetween, a first luma tree for the luma component of a first CU, a first chroma tree for the chroma components of the first CU, a second luma tree for the luma component of a second CU, and a second chroma tree for the chroma components of the second CU.

[0044] These conventional approaches have their limits and problems.

[0045] For example, coding according to the tree coding approaches described above must be completed one CTU at a time. Specifically, coding of the luma and chroma components of a CTU must be completed before proceeding to the next CTU. Furthermore, a frame (or a portion thereof that can be coded independently of another portion, such as a slice, a tile, or a segment) is a combination of both its luma and chroma components. Thus, to use the frame as a reference frame, both of luma and chroma components must have been decoded (e.g., reconstructed) first. Therefore, coding of a subsequent frame that relies on the reference frame may be stalled (e.g., available processors sit idle) until the reference frame is available. To illustrate, in the case of hierarchical coding structures, frames (or portions thereof) at lower temporal levels in the coding structure can be bottlenecks in parallel coding or multithreading.

[0046] As another problem, small chroma blocks are known to be hardware unfriendly. Codecs may introduce special handling for small chroma blocks in the case of single tree. However, the single tree may introduce many comer cases that are specifically handled via a sub-optimal hardware design and custom circuitry.

[0047] Additionally, the traditional approaches limit the level of parallelism that can be exploited by codecs (e.g., in parallel encoding and decoding). Luma CUs and chroma CUs are processed one after the other, even when there is no interdependence between them. The situation may be further aggravated on the decoder side, where luma and chroma cannot be parsed concurrently due to the interleaved signaling, which in turn negatively impacts throughput, especially for high-resolution formats, such as 8K video.

[0048] Video compression with separated luma and chroma planes, as described herein, separates the luma CTUs and chroma CTUs into separate planes within a compressed bitstream. Each of the planes can be separately coded. As such, at least the coding of luma CTUs can proceed independent of the coding of chroma CTUs. Accordingly, parallelism can be improved over the traditional approaches. Furthermore, no special hardware/circuitry is required in hardware implementations as no special cases need be handled. Additionally, in low-bandwidth environments, it is possible for a decoder to decode the luma signal and to ignore the chroma signal, therewith resulting in a monochrome video stream.

[0049] Moreover, separating the luma and chroma planes can improve the prediction quality for certain prediction modes whereby chroma blocks are predicted based on or from luma blocks. Where chroma blocks are predicted from luma blocks or images, as further described herein, the prediction can use post-filtered luma blocks or images. A post-filtered luma image can provide a better (e.g., improved or enhanced) luma signal over an only reconstructed set of luma blocks. Traditionally, such prediction is based on only reconstructed luma blocks or images since, in the case of luma CTU and chroma CTU interleaving, it is not possible to loop filter and then predict chroma since loop filtering is typically applied to both luma and chroma signals.

[0050] According to an implementation, a compressed bitstream is received. The compressed bitstream includes, with respect to a group of coding tree units, a luma plane followed by a chroma plane. The luma plane includes data relating to luma pixel values and the chroma plane includes data relating to chroma pixel values. A luma block of the luma plane is decoded based on a luma reference block included in another luma plane. A chroma block of the chroma plane is decoded based on a reference luma block, wherein the chroma block is collocated with the luma block. In an example, the blocks of the chroma plane can only be coded in merge/skip mode.

[0051] While the description herein is mainly described with respect to one color space, namely, the YUV color space, the disclosure is not so limited. Although terminology such as luma and chroma planes, or Y/U/V planes are used, the disclosure herein can be easily extended to other color formats, such as the YCoCg, YCbCr, RGB, or other color spaces. Additionally, according to the teachings herein, more planes can be included in a compressed bitstream. For example, the planes may include a transparency plane in addition to the luma and chroma planes or a depth plane in addition to the red, green, and blue planes.

[0052] Further details of techniques for video compression with separated luma and chroma planes are described herein with initial reference to a system in which they can be implemented. FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such as that described in FIG. 2. However, other implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0053] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0054] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0055] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and/or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, e.g., a video streaming protocol based on the Hypertext Transfer Protocol (HTTP).

[0056] When used in a video conferencing system, for example, the transmitting station 102 and/or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a video conference server (e.g., the transmitting station 102) to decode and view and further encodes and transmits his or her own video bitstream to the video conference server for decoding and viewing by other participants.

[0057] FIG. 2 is a block diagram of an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106 of FIG. 1. The computing device 200 can be in the form of a computing system including multiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

[0058] A processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. For example, although the disclosed implementations can be practiced with one processor as shown (e.g., the processor 202), advantages in speed and efficiency can be achieved by using more than one processor.

[0059] A memory 204 in computing device 200 can be a read only memory (ROM) device or a random-access memory (RAM) device in an implementation. However, other suitable types of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the processor 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the processor 202 to perform the techniques described herein. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described herein. The computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device.

Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0060] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, the display can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0061] The computing device 200 can also include or be in communication with an image-sensing device 220, for example, a camera, or any other image-sensing device 220 now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image-sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0062] The computing device 200 can also include or be in communication with a soundsensing device 222, for example, a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound-sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0063] Although FIG. 2 depicts the processor 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the processor 202 can be distributed across multiple machines (wherein individual machines can have one or more processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0064] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individual frames, for example, a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 can be subsets of frames that permit parallel processing, for example. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can include a luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0065] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise noted, the terms block and macroblock are used interchangeably herein. [0066] FIG. 4 is a block diagram of an encoder 400 according to implementations of this disclosure. The encoder 400 can be implemented, as described above, in the transmitting station 102, such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0067] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra/inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0068] When the video stream 300 is presented for encoding, respective adjacent frames 304, such as the frame 306, can be processed in units of blocks. At the intra/inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intraprediction) or inter- frame prediction (also called inter-prediction). In any case, a prediction block can be formed. In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0069] Next, the prediction block can be subtracted from the current block at the intra/inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated. [0070] The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0071] The reconstruction path (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below with respect to FIG. 5) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are similar to functions that take place during the decoding process (described below with respect to FIG. 5), including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra/inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0072] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform-based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In some implementations, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0073] FIG. 5 is a block diagram of a decoder 500 according to implementations of this disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0074] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra/inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0075] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra/inter prediction stage 508 to create the same prediction block as was created in the encoder 400 (e.g., at the intra/inter prediction stage 402).

[0076] At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, the deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as the output video stream 516. The output video stream 516 can also be referred to as a decoded video stream, and the terms will be used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 can produce the output video stream 516 without the deblocking filtering stage 514.

[0077] FIG. 6 is a block diagram of a representation of a portion 600 of a frame, such as the frame 306 of FIG. 3, according to implementations of this disclosure. As shown, the portion 600 of the frame includes four 64x64 blocks 610, which may be referred to as superblocks, in two rows and two columns in a matrix or Cartesian plane. A superblock can have a larger or a smaller size. For example, a superblock can be 128x128. A superblock can also be referred to as a coding tree block (CTB). While FIG. 6 is explained with respect to a superblock of size 64x64, the description is easily extendable to larger (e.g., 128x128) or smaller superblock sizes.

[0078] In an example, a superblock can be a basic or maximum coding unit (CU). Each superblock can include four 32x32 blocks 620. Each 32x32 block 620 can include four

16x16 blocks 630. Each 16x16 block 630 can include four 8x8 blocks 640. Each 8x8 block 640 can include four 4x4 blocks 650. Each 4x4 block 650 can include 16 pixels, which can be represented in four rows and four columns in each respective block in the Cartesian plane or matrix. The pixels can include information representing an image captured in the frame, such as luminance information, color information, and location information. In an example, a block, such as a 16xl6-pixel block as shown, can include a luminance block 660, which can include luminance pixels 662; and two chrominance blocks 670/680, such as a U or Cb chrominance block 670, and a V or Cr chrominance block 680. The chrominance blocks 670/680 can include chrominance pixels 690. For example, the luminance block 660 can include 16x16 luminance pixels 662, and each chrominance block 670/680 can include 8x8 chrominance pixels 690, as shown. Although one arrangement of blocks is shown, any arrangement can be used. Although FIG. 6 shows NxN blocks, in some implementations, NxM, where N^M, blocks can be used. For example, 32x64 blocks, 64x32 blocks, 16x32 blocks, 32x16 blocks, or any other size blocks can be used. In some implementations, Nx2N blocks, 2NxN blocks, or a combination thereof can be used.

[0079] In some implementations, video coding can include ordered block-level coding. Ordered block-level coding can include coding blocks of a frame in an order, such as rasterscan order, wherein blocks can be identified and processed starting with a block in the upper left comer of the frame, or a portion of the frame, and proceeding along rows from left to right and from the top row to the bottom row, identifying each block in turn for processing. For example, the superblock in the top row and left column of a frame can be the first block coded, and the superblock immediately to the right of the first block can be the second block coded. The second row from the top can be the second row coded, such that the superblock in the left column of the second row can be coded after the superblock in the rightmost column of the first row.

[0080] In an example, coding a block can include using quad-tree coding, which can include coding smaller block units with a block in raster-scan order. The 64x64 superblock shown in the bottom-left corner of the portion of the frame shown in FIG. 6, for example, can be coded using quad-tree coding in which the top-left 32x32 block can be coded, then the top-right 32x32 block can be coded, then the bottom-left 32x32 block can be coded, and then the bottom-right 32x32 block can be coded. Each 32x32 block can be coded using quad-tree coding in which the top-left 16x16 block can be coded, then the top-right 16x16 block can be coded, then the bottom-left 16x16 block can be coded, and then the bottom-right 16x16 block can be coded. Each 16x16 block can be coded using quad-tree coding in which the top-left 8x8 block can be coded, then the top-right 8x8 block can be coded, then the bottom-left 8x8 block can be coded, and then the bottom-right 8x8 block can be coded. Each 8x8 block can be coded using quad-tree coding in which the top-left 4x4 block can be coded, then the topright 4x4 block can be coded, then the bottom-left 4x4 block can be coded, and then the bottom-right 4x4 block can be coded. In some implementations, 8x8 blocks can be omitted for a 16x16 block, and the 16x16 block can be coded using quad-tree coding in which the top-left 4x4 block can be coded, and then the other 4x4 blocks in the 16x16 block can be coded in raster-scan order.

[0081] In an example, video coding can include compressing the information included in an original, or input, frame by omitting some of the information in the original frame from a corresponding encoded frame. For example, coding can include reducing spectral redundancy, reducing spatial redundancy, reducing temporal redundancy, or a combination thereof.

[0082] In an example, reducing spectral redundancy can include using a color model based on a luminance component (Y) and two chrominance components (U and V or Cb and Cr), which can be referred to as the YUV or YCbCr color model or color space. Using the YUV color model can include using a relatively large amount of information to represent the luminance component of a portion of a frame and using a relatively small amount of information to represent each corresponding chrominance component for the portion of the frame. For example, a portion of a frame can be represented by a high-resolution luminance component, which can include a 16x16 block of pixels, and by two lower resolution chrominance components, each of which representing the portion of the frame as an 8x8 block of pixels. A pixel can indicate a value (e.g., a value in the range from 0 to 255) and can be stored or transmitted using, for example, eight bits. Although this disclosure is described with reference to the YUV color model, any color model can be used.

[0083] As mentioned, a block can be coded using tree coding. In an example, the tree can be a quad-tree. FIG. 7 is a block diagram of an example 700 of a quad-tree representation of a block according to implementations of this disclosure. The example 700 includes a block 702. As mentioned above, the block 702 can be referred to as a superblock or a CTB. The example 700 illustrates a partition of the block 702. However, the block 702 can be partitioned differently, such as by an encoder (e.g., the encoder 400 of FIG. 4).

[0084] The example 700 illustrates that the block 702 is partitioned into four blocks, namely, blocks 702-1, 702-2, 702-3, and 702-4. The block 702-2 is further partitioned into blocks 702-5, 702-6, 702-7, and 702-8. As such, if, for example, the size of the block 702 is NxN (e.g., 128x128), then the blocks 702-1, 702-2, 702-3, and 702-4 are each of size N/2xN/2 (e.g., 64x64), and the blocks 702-5, 702-6, 702-7, and 702-8 are each of size N/4xN/4 (e.g., 32x32). If a block is partitioned, it is partitioned into four equally sized, nonoverlapping square sub-blocks.

[0085] A quad- tree data representation is used to describe how the block 702 is partitioned into sub-blocks, such as blocks 702-1, 702-2, 702-3, 702-4, 702-5, 702-6, 702-7, and 702-8. A quad-tree 703 of the partition of the block 702 is shown. Each node of the quadtree 703 is assigned a flag of “1” if the node is further split into four sub-nodes and assigned a flag of “0” if the node is not split. The flag can be referred to as a split bit (e.g., 1) or a stop bit (e.g., 0) and is coded in a compressed bitstream. In a quad-tree, a node either has four child nodes or has no child nodes. A node that has no child nodes corresponds to a block that is not split further. Each of the child nodes of a split block corresponds to a sub-block.

[0086] In the quad-tree 703, each node corresponds to a sub-block of the block 702. The sub-block is shown between parentheses. For example, a node 704-1, which has a value of 0, corresponds to the block 702-1.

[0087] A root node 704-0 corresponds to the block 702. As the block 702 is split into four sub-blocks, the value of the root node 704-0 is the split bit (e.g., 1). At an intermediate level, the flags indicate whether a sub-block of the block 702 is further split into four sub- subblocks. In this case, a node 704-2 includes a flag of “1” because the block 702-2 has been split into the blocks 702-5, 702-6, 702-7, and 702-8. Each of nodes 704-1, 704-3, and 704-4 includes a flag of “0” because the corresponding blocks are not split. As nodes 704-5, 704-6, 704-7, and 704-8 are at a bottom level, no flag of “0” or “1” is necessary for those nodes because of corresponding CUs. That the blocks 702-5, 702-6, 702-7, and 702-8 are not split further can be inferred from the absence of additional flags corresponding to these blocks.

[0088] The quad-tree data representation for the quad- tree 703 can be represented by the binary data of “10100,” where each bit represents a node 704 of the quad-tree 703. The binary data indicates the partitioning of the block 702 to the encoder and decoder. The encoder can encode the binary data in a compressed bitstream, such as the compressed bitstream 420 of FIG. 4, in a case where the encoder needs to communicate the binary data to a decoder, such as the decoder 500 of FIG. 5.

[0089] The blocks corresponding to the leaf nodes of the quad- tree 703 can be used as the bases for prediction. That is, prediction can be performed for each of the blocks 702-1, 702-5, 702-6, 702-7, 702-8, 702-3, and 702-4, referred to herein as coding blocks. As mentioned with respect to FIG. 6, the coding block can be a luminance block or a chrominance block. It is noted that, in an example, the superblock partitioning can be determined with respect to luminance blocks. The same partition can be used with the chrominance blocks.

[0090] A prediction type (e.g., intra- or inter-prediction) is determined at the coding block (e.g., a block 702-1, 702-5, 702-6, 702-7, 702-8, 702-3, or 702-4) level. That is, a coding block is the decision point for prediction.

[0091] FIG. 8A is a block diagram of an example 800 of recursive partitioning of a coding block. The example 800 includes a coding block 802. Inter- or intra-prediction is performed with respect to the coding block 802. That is, the coding block 802 can be partitioned (e.g., divided, split, or otherwise partitioned) into one or more prediction units (PU) according to a partition type, such as one of the partition types described herein. Each PU can be predicted using inter- or intra-prediction. In an example, the process described with respect to the example 800 can be performed (e.g., implemented) by an intra/inter- prediction stage, such as the intra/inter-prediction stage 402 of the encoder 400 of FIG. 4. It is noted that while certain partitions are described with respect to FIG. 8, these partitions are meant to be illustrative and non-limiting. Other partition types are possible.

[0092] The coding block 802 can be a chrominance block. The coding block 802 can be a luminance block. In an example, a partition is determined for a luminance block, and a corresponding chrominance block uses the same partition as that of the luminance block. In another example, a partition of a chrominance block can be determined independently of the partition of a luminance block.

[0093] The example 800 illustrates a recursive partition search (performed at an encoder) of the coding block 802. The recursive search is performed to determine the partition that results in the optimal rate-distortion (RD) cost. An RD cost can include the cost of encoding both the luminance and the chrominance blocks corresponding to a block.

[0094] The example 800 illustrates four partition types that may be available at an encoder. A partition type 804 (also referred to herein as the PARTITION_SPLIT partition type and partition- split partition type) splits the coding block 802 into four equally sized square sub-blocks. For example, if the coding block 802 is of size NxN, then each of the four sub-blocks of the PARTITION_SPLIT partition type is of size N/4xN/4. Each of the four sub-blocks resulting from the partition type 804 is not itself a prediction unit/block.

[0095] A partition type 806 (also referred to herein as the PARTITION_VERT partition type) splits the coding block 802 into two adjacent rectangular prediction units, each of size NxN/2. A partition type 808 (also referred to herein as the PARTITION_HORZ partition type) splits the coding block 802 into two adjacent rectangular prediction units, each of size N/2xN. A partition type 810 (also referred to herein as the PARTITION_NONE partition type and partition-none partition type) uses one prediction unit for the coding block 802 such that the prediction unit has the same size (i.e., NxN) as the coding block 802.

[0096] For brevity, a partition type may simply be referred to herein by its name only. For example, instead of using “the PARTITION_VERT partition type,” “the PARTITION_VERT” may be used instead. As another example, instead of “the partition- none partition type,” “the partition-none” may be used. Additionally, uppercase or lowercase letters may be used to refer to partition type names. As such, “PARTITION_VERT” and “partition- vert” refer to the same partition type.

[0097] Except for the partition type 804, none of the other partitions can be split further. As such, the partition types 806-810 can be considered end points. Each of the sub-blocks of a partition (according to a partition type) that is not an end point can be further partitioned using the available partition types. As such, partitioning can be further performed for square coding blocks. The sub-blocks of a partition type that is an end point are not partitioned further. As such, further partitioning is possible only for the sub-blocks of the PARTITION_SPLIT partition type.

[0098] As mentioned above, to determine the minimal RD cost for the coding block 802, the coding block is partitioned according to the available partition types, and a respective cost (e.g., an RD cost) of encoding the block based on each partition is determined. The partition type resulting in the smallest RD cost is selected as the partition type to be used for partitioning and encoding the coding block.

[0099] The RD cost of a partition is the sum of the RD costs of each of the sub-blocks of the partition. For example, the RD cost associated with the PARTITION_VERT (i.e., the partition type 806) is the sum of the RD cost of a sub-block 806A and the RD cost of a subblock 806B. The sub-blocks 806 A and 806B are prediction units. In an example, identifiers, such as identifiers 0, 1, 2, and 4, can be associated, respectively, with the PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT, and PARTITION_SPLIT. Other identifiers are possible. Other ways of communicating the partition type to a decoder are possible.

[0100] To determine an RD cost associated with a prediction block, an encoder can predict the prediction block using at least some of the available prediction modes (i.e., available inter- and intra-prediction modes). In an example, for each of the prediction modes, a corresponding residual is determined, transformed, and quantized to determine the distortion and the rate (in bits) associated with the prediction mode. As mentioned, the partition type resulting in the smallest RD cost can be selected. Selecting a partition type can mean, inter alia, encoding in a compressed bitstream, such as the compressed bitstream 420 of FIG. 4, the partition type. Encoding the partition type can mean encoding an identifier corresponding to the partition type. Encoding the identifier corresponding to the partition type can mean entropy encoding, such as by the entropy encoding stage 408 of FIG. 4, the identifier.

[0101] To determine the RD cost corresponding to the PARTITION_SPLIT (i.e., the partition type 804), a respective RD cost corresponding to each of the sub-blocks, such as a sub-block 812, is determined. As the sub-block 812 is a square sub-block, the sub-block 812 is further partitioned according to the available partition types to determine a minimal RD cost for the sub-block 812. As such, the sub-block 812 is further partitioned as shown with respect to partitions 814. As the sub-blocks of a partition 816 (corresponding to the PARTITION_SPLIT) are square sub-blocks, the process repeats for each of the sub-blocks of the partition 816, as illustrated with an ellipsis 818, until each of a smallest square sub-block size is reached. The smallest square sub-block size corresponds to a block size that is not partitionable further. In an example, the smallest square sub-block size, for a luminance block, is a 4x4 block size.

[0102] As such, determining an RD cost of a square block can be regarded as a bottom-up search. That is, for example, to determine the RD cost of a PARTITION_SPLIT of a 16x16 coding block, the RD cost of each of the four 8x8 sub-blocks is determined; to determine the RD cost of a PARTITION_SPLIT of a 4x4 coding block, the RD cost of each of the four 4x4 sub-blocks is determined. As such, a square block can be recursively partitioned, based on a quad-tree partitioning, into sub-blocks using the partition-split type.

[0103] As mentioned above, more partition types than those described with respect to FIG. 8A can be available at a codec. FIG. 8B is a block diagram of an example 820 of extended partition types of a coding block according to implementations of this disclosure. The term “extended” in this context can mean “additional.”

[0104] A partition type 822 (also referred to herein as the PARTITION_VERT_A) splits an NxN coding block into two horizontally adjacent square blocks, each of size N/2xN/2, and a rectangular prediction unit of size NxN/2. A partition type 828 (also referred to herein as the PARTITION_VERT_B) splits an NxN coding block into a rectangular prediction unit of size NxN/2 and two horizontally adjacent square blocks, each of size N/2xN/2.

[0105] A partition type 824 (also referred to herein as the PARTITION_HORZ_A) splits an NxN coding block into two vertically adjacent square blocks, each of size N/2xN/2, and a rectangular prediction unit of size N/2xN. A partition type 830 (also referred to herein as the PARTITION_HORZ_B) splits an NxN coding block into a rectangular prediction unit of size N/2xN and two vertically adjacent square blocks, each of size N/2xN/2.

[0106] A partition type 826 (also referred to herein as the PARTITION_VERT_4) splits an NxN coding block into four vertically adjacent rectangular blocks, each of size NxN/4. A partition type 832 (also referred to herein as the PARTITION_HORZ_4) splits an NxN coding block into four horizontally adjacent rectangular blocks, each of size N/4xN.

[0107] As mentioned above, a recursive partition search (e.g., based on a quad-tree partitioning) can be applied to square sub-blocks, such as sub-blocks 822A, 822B, 824A, 824B, 828A, 828B, 830A, and 830B.

[0108] Identifiers can be associated with each of the partition types of the example 820. In an example, identifiers 4-9 can be associated, respectively, with the PARTITION_HORZ_A, PARTITION_HORZ_B, PARTITION_VERT_A, PARTITION_VERT_B, PARTITION_HORZ_4, and PARTITION_VERT_4. Other identifiers are possible.

[0109] As shown in the example 820, instead of the four possible partition types of the example 800, 10 possible partition types (the partition types of the example 800 and the partition types of the example 820) can be available at an encoder. The complexity of an encoder that uses the 10 partition types can be 2.5 times that of an encoder that uses only the four partition types.

[0110] FIG. 9 is a flowchart of a technique 900 for decoding coding tree units structured into planes of color components. The technique 900 can be implemented, for example, as a software program that may be executed by computing devices such as transmitting station 102 or receiving station 106. The software program can include machine-readable instructions that may be stored in a memory such as the memory 204 or the secondary storage 214, and that, when executed by a processor, such as the processor 202, may cause the computing device to perform the technique 900. The technique 900 may be implemented in whole or in part in the intra/inter prediction stage 508 of the decoder 500 of FIG. 5. The technique 900 can be implemented using specialized hardware or firmware. Multiple processors, memories, or both, may be used.

[0111] At 902, a compressed bitstream is received. The compressed bitstream can be the compressed bitstream 420 of FIG. 4. With respect to a group of coding tree units, the compressed bitstream is structured into planes corresponding to different color components. A group of coding tree units can be a tile of the current frame, a segment of the current frame, a slice of the current frame, or the current frame as a whole. More broadly, the group of coding tree units can be any grouping of blocks that can be decoded independently from and, if the decoder is so capable, in parallel with other groups of coding tree units.

[0112] In decoding blocks in the different color component planes, temporal reference can be at the plane level. A reference frame can be considered to be a combination of related luma plane(s), chroma plane(s), and coding information plane (coding mode and motion vector). As further described herein, in an example, a luma plane (e.g., blocks therein) can only be predicted from luma planes while a chroma plane (e.g., blocks therein) can be predicted from both luma and chroma planes. Said another way, the decoding of a luma plane can be completely independent from the decoding of the collocated chroma plane(s). In an example, when using prediction from luma plane to chroma plane (such as described with respect to cross -component prediction), the prediction can use the post-filtered (e.g., not only a reconstructed) luma plane. That is, the luma plane that results after all the loop filtering is performed with respect to the reconstructed luma plane is used. In an example, the motion information in the coding information plane can be derived during the decoding of the related luma planes. Suh motion information cannot be changed during the decoding of the collocated chroma planes.

[0113] FIG. 10 illustrates examples of compressed bitstreams structured into color component planes. Generally, and with respect to a group of coding tree units, the compressed bitstream includes a first color component plane followed by a second color component plane. Several possible arrangements are shown in FIG. 10. However, additional arrangements of color component planes are also possible, and the disclosure is not limited to those shown in FIG. 10. To be clear, a plane corresponding to a color component includes pixel data (e.g., residuals or transform coefficients therefor) for that color component. As such, a luma plane includes only luma pixels (e.g., data related to luma pixel values) and a chroma plane includes only chroma pixels (e.g., data related to chroma pixel values).

[0114] An arrangement 1002 illustrates that the compressed bitstream includes a coding information plane 1004, followed by a luma plane 1006, which is then followed by a chroma plane 1008. The luma plane 1006 includes coding tree units corresponding to a group of coding tree units. The chroma plane 1008 can include the coding tree units of the chroma U and chroma V color components.

[0115] The coding information plane 1004 can include motion information and, optionally, coding mode information. For example, the coding information plane 1004 can include motion information of reference frames or blocks. To illustrate, the coding information plane 1004 may include parameters of a global motion model usable for decoding at least one block of the luma plane 1006 or at least one block of the chroma plane 1008. More broadly, the coding information plane 1004 includes non-pixel information (i.e., data unrelated to pixel values). The coding information plane 1004 can include, per block, the coding mode information (e.g., a motion vector and a reference frame for that block). The coding information plane 1004 may traditionally be included in block headers and include coding mode information of the blocks. The motion information included in the coding information plane, which may traditionally be interleaved with the luma partition information, is required for determining which block the motion information should be applied to.

[0116] By separating the coding information plane 1004 from the luma plane 1006 and the chroma plane 1008, the coding information can be shared by the luma blocks of the luma plane 1006 and the blocks of the chroma plane 1008. As such, after the coding information plane 1004 is read and stored by the decoder, the decoder can proceed to decode the blocks of the luma plane 1006 and the chroma plane 1008 with reference to the stored information. [0117] In an arrangement 1010, the compressed bitstream can include a luma and coding information plane 1012 and a chroma plane 1014. In the arrangement 1010, at least some of the coding information may be included in block headers. That is, the coding mode information for a luma block may be included in a header of the luma block. In the arrangement 1010, any coding mode information required for blocks of the chroma plane 1014 are obtained from the coding mode information of the collocated blocks in the luma and coding information plane 1012. As such, the motion information in the coding information plane is derived during the decoding of the related luma planes. Such motion information cannot be changed during the decoding of the collocated chroma blocks in the chroma plane 1014. That is, for example, blocks in the chroma plane cannot have different motion vectors than their collocated luma blocks in the luma plane. A decoder can derive the motion information of chroma blocks from their collocated luma blocks. As such, the motion information of a luma block may be used for a corresponding chroma block or may not be used; but the compressed bitstream cannot include and the decoder does not change (e.g., modify, use different) motion information.

[0118] In an arrangement 1016, the compressed bitstream can include a luma and coding information plane 1018, followed by a chroma U plane 1020, which is then followed by a chroma V plane. In an arrangement 1024, the compressed bitstream can include a coding information plane 1026, followed by a luma plane 1028, followed by a chroma U plane 1030, which is then followed by a chroma V plane 1032.

[0119] Referring again the FIG. 9, and consistent with the description of FIG. 10, the compressed bitstream includes, with respect to the group of coding tree units, a luma plane that includes data related to luma pixel values followed by a chroma plane that includes data related to chroma pixel values. In an example, the luma plane and the chroma plane can be parsed independently.

[0120] In an example, the luma plane includes a first tree structure corresponding to a partitioning of the blocks of the luma plane and the chroma plane includes a second tree structure corresponding to a partitioning of the blocks of the chroma plane. The structure of the first tree can be different from the structure of the second tree. For example, the first tree may be a multi-type tree and the second tree may be a quaternary-tree, vice versa, or some other types of trees. In an example, tree scanning order may be different in different planes. As such, a first scanning order can be used with the first tree and a second scanning order that is different from the first scanning order can be for scanning the second tree.

[0121] CTU scanning orders may be different in different planes. As such, a first scanning order may be used to scan coding tree units of the luma plane and a second scanning order that is different from the first scanning order may be used to scan coding tree units of the chroma plane. To illustrate, and without limitations, the first scanning order may be a raster scan order from left to right and the second scanning order may be a raster scan order from right to left.

[0122] At 904, a luma block of the luma plane is decoded based on a luma reference block included in another luma plane. That is, a block in the luma plane can only be predicted from data in other luma planes. As such, if decoding the luma block uses a motion vector and a reference frame, then only the luma portion of the reference frame need be available before decoding the luma block can begin. That is, decoding the luma block can begin as soon as the luma component of the reference frame is available. Decoding the luma block need not wait until both the luma and chroma components of the reference frame are available. Said another way, decoding of a luma plane (i.e., luma blocks therein) can be completely independent from the decoding of the collocated chroma plane(s) (i.e., chroma block(s) therein).

[0123] At 906, a chroma block of the chroma plane is decoded based on at least one of a reference luma block or a reference chroma block. The chroma block is collocated with the luma block. That the chroma block is collocated with the luma includes that the chroma block may be shifted from the luma block based on a shift value. The shift value may include a horizontal shift, a vertical shift, or both. The vertical shift and the vertical shift may each be less than one (1) luma pixel in the horizontal and the vertical directions, respectively, from the collocated luma block.

[0124] In an example, the chroma block can be predicted from either of the luma or the chroma planes, such as traditionally done with respect to inter-prediction. The chroma block can be predicted from another chroma block, such as according to known and conventional inter prediction (chroma is temporally predicted from a chroma block). In another example, the chroma block can be predicted from a luma block, which may be a collocated luma block. The chroma block can be predicted from the luma block as described with respect to at least one of FIG. 11 (with FIG. 12) or FIG. 13.

[0125] In an example, coding units in the chroma plane are only coded in merge/skip modes. As such, coding (e.g., prediction) modes need not be included (e.g., signaled) in the compressed bitstream for blocks of the chroma plane. Thus, CUs in the chroma plane are only coded in merge/skip modes without explicitly signaling mode types (e.g., inter, intra, or intra block copy (IBC)) for the blocks of the chroma plane. As such, to decode the chroma block, a list of candidate merge blocks may be generated for the chroma block and an index of the candidate to be used as the merge candidate can be decoded from the compressed bitstream. The list of the candidate merge blocks can include one or more of temporal prediction candidates, cross -component candidates, IBC candidates, or spatial candidates. [0126] With respect to temporal prediction candidates, the candidates can be based on the collocated luma blocks. That is, the candidate motion information can be from the collocated luma blocks. The motion information can be at sub-block level. For example, each sub-block of the collocated luma block may have different motion information. In an example, if any of the sub-blocks of the luma block are not inter-predicted, then the temporal prediction candidates are not generated for the chroma block. That is, the set of the candidate merge blocks would not include temporal prediction candidates. As such, the list of merge candidates can include temporal prediction candidates where each of the temporal prediction candidates can be based on a respective motion vector of a sub-block of the luma block of the luma plane.

[0127] After temporal prediction, the chroma prediction may be enhanced by crosscomponent filtering, where the filtering coefficients can be derived from the chroma and luma reconstructions of a neighboring template of the current block. That is, if one of the temporal candidates were to be selected for the chroma block, then after generating a prediction block based on the temporal candidate (e.g., based on the motion information of the temporal candidate block), the prediction block (i.e., the chroma prediction) may be enhanced by cross-component filtering. The filter coefficients may also be derived from luma and chroma temporal predictions. As such, the technique 900 can include obtaining an intermediate prediction block corresponding to one of the temporal prediction candidates; and applying a filter to the intermediate prediction block to obtain a final prediction block. In an example, the cross-component filtering can be as described with respect to FIGS. 11-12. In an example, whether cross-component filtering is used can be conditionally signaled in the compressed bitstream. That is, the technique 900 can decode a syntax element from the compressed bitstream indicating whether cross-component filtering is to be applied.

[0128] In an example, a cross-component prediction from luma may be generated in addition to chroma temporal prediction. A final chroma prediction may be obtained as a weighted sum of the chroma temporal prediction and the cross-component prediction. As such, the technique 900 can include obtaining a first intermediate prediction block corresponding to one of the temporal prediction candidates. A second intermediate prediction block can be obtained based on a cross -component prediction from the luma block. The first intermediate prediction block and the second intermediate prediction block can be combined (e.g., weighted) to obtain a final prediction block. In an example, the prediction from luma can be generated using the cross-component prediction described with respect to FIG. 13. In another example, the prediction from luma can be generated using the cross-component prediction described with respect to FIG. 14.

[0129] With respect to cross-component candidates, predictions from filtered collocated luma reconstructions can be generated. Filtering coefficients for each of the predictions can be derived from the chroma and luma reconstructions of a neighboring template of the chroma block. Different (small) offsets may be further used with each of the predictions to indicate the relative position between luma and chroma samples so that different chroma location types may be supported. Different filters may be used for the different candidates (i.e., the different predictions). Alternatively, the different filters may be selected based on (e.g., according to) the respective offset value used. The cross-component candidates can be generated as described with respect to, or consistent with, FIGS. 11-14.

[0130] With respect to the IBC candidates, predictions from previously reconstructed chroma samples within the current chroma plane can be generated. Accordingly, the list of merge candidates can include at least one candidate that is based on an intra block copy mode. In an example, after an IBC prediction is generated, the chroma prediction may be further enhanced (e.g., filtered) by cross -component filtering, as described herein. As such, the at least one candidate can be obtained based on filter coefficients obtained via a crosscomponent filtering operation. That is, if an IBC candidate is selected for decoding the chroma block, the generated prediction block can be further enhanced based on a crosscomponent filtering operation.

[0131] With respect to spatial candidates, predictions from spatial neighbors can be generated using a pre-defined list of intra-prediction modes. In an example, a DC chroma prediction, a vertical chroma prediction, a horizontal chroma prediction, and a planar chroma prediction can be generated. In an example, a palette chroma prediction can also be generated.

[0132] FIG. 11 illustrates an example 1100 of cross-component filtering. The example 1100 illustrates pixels that are filled with different patterns. The example 1100 is used to illustrate a current block template and is also used to illustrate a reference block template. [0133] When used to describe a current block template, a block 1108 illustrates a current block (i.e., the block being decoded). While the block 1108 is shown as being of size 8x4, the disclosure is not so limited. The block 1108 can be of any other size. Pixels filled with a pattern 1102 are pixels of the current block of a current frame. Pixels filled with a pattern 1106 (or a subset thereof, as further described herein) illustrate reconstructed pixels of the current frame. Pixels filled with a pattern 1104 are pixels that are not available and may contain a padding value (i.e., are set to a padding value). Depending on the neighborhood used for a filter, one or more pixels used by the filter may not be available (such as because these pixels are outside the frame boundary or are outside a largest coding unit that includes the block 1108). As such, a padding value may be used (e.g., assumed) for such pixels.

[0134] When used to describe a reference block template, the block 1108 illustrates a reference block in a reference frame. Again, while the block 1108 is shown as being of size 8x4, the disclosure is not so limited. The block 1108 can be of a size corresponding to the size of the current block. As such, pixels filled with the pattern 1102 are pixels of the reference block of a reference frame. Pixels filled with the pattern 1106 (or a subset thereof, as further described herein) illustrate reconstructed pixels of the reference frame. Pixels filled with the pattern 1104 are pixels that are not available and may contain a padding value.

[0135] The template may include a top region 1110 that may include 1 to N (where N>1) rows of pixels. The template may include a top-right region 1112 that includes 1 to N rows. The template may include a left region 1114 of 1 to M (where M>1) columns of pixels. The template may include a bottom- left region 1116 of 1 to M (where M>1) columns of pixels. [0136] In an example, N=M. In an example, if the current block is a luma block, then the template can be 4-sample wide. If the current block is a chroma block, the template (i.e., a chroma template) may be based on the chroma color format. For example, for 4:4:4 content, the chroma template can also be 4-sample wide; and for 4:2:0 or 4:2:2 color formats, the chroma template can be 2-sample wide. In an example, when the top-right region 1112 is available, only a 4x4 luma block at the top-right is included in the template. Similarly, if the bottom-left region 1116 is available, only a 4x4 luma block at bottom-right is included in the template. The chroma template can be adjusted accordingly based on the chroma color format. In another example, the top template may always be 1 -sample wide for both luma and chroma while the left template may be 4-sample wide for luma.

[0137] In an example, the filter coefficients include at least two coefficients. In an example, the filter coefficients include more than two coefficients for at least one of the color components (i.e., at least of the luma or the chroma component). In an example, the number (i.e., cardinality) of the filter coefficients can be decoded from the compressed bitstream. For example, an indicator of the number of filter coefficients can be decoded from the compressed bitstream. That is, if the indicator of the number of coefficient is the first value (e.g., 0), then no filtering is performed on the prediction block. If the indicator of the number of the fdter coefficients is a second value (e.g., 1), then two fdter coefficients are derived; and if the indicator of the number of filter coefficients is a third value (e.g., 2), then more than two filter coefficients are derived.

[0138] In an example, the filter can be a convolutional filter. The filter coefficients can be obtained by minimizing an error metric between the first reconstructed pixels and the second reconstructed pixels. The error metric can be a mean square error (MSE) between pixel values of the respective reconstructed pixels. The error can be a sum of absolute differences (SAD) error between the pixel values of the reconstructed pixels. Any other suitable error metric can be used.

[0139] In an example, the number of coefficients to be obtained depends on which pixels within the neighborhood of the intermediate prediction pixel to which the filter is to be applied are used in the filtering. The pixels within the neighborhood of an intermediate prediction pixel that are used for filtering are referred to herein as at least a subset of pixels of the neighborhood.

[0140] FIG. 12 illustrates an example 1200 of a neighborhood of an intermediate pixel 1202 of an intermediate prediction block. The example 1200 illustrates a 3x3 neighborhood. However, the neighborhood can be larger or smaller, rectangular, or some other shape (e.g., diamond). The example 1200 illustrates that pixels 1204 to 1210 (i.e., pixels to the north,

- l- east, south, and west of the intermediate pixel 1202, respectively) are used in the filtering. As such, the filter coefficients include at least five coefficients: one coefficient to be used with each of the pixels 1202 to 1210.

[0141] As such, the filter is a 5-tap filter and the prediction pixel corresponding to the intermediate pixel 1202 can be obtained using equation (1), where Ci (i=0,...,4) are the filter coefficients, pred is the filter pixel of the final prediction block. Equation (1) is shown as further including a constant term (i.e., cs), which may also be derived and used ins some implementations . pred = c₀C + c₄N + c₂S + c₃E + c₄W + c₅ (1)

[0142] In an example, one or more but not all filter coefficients may be further refined after being derived. As such, the obtained filter coefficients may be considered to be predicted filter coefficients. The difference (i.e., a coefficient refinement value) between a predicted filter coefficient and the actual value of the filter coefficient may be signaled in the compressed bitstream. As such, obtaining the filter coefficients for the filter can include obtaining a predicted filter coefficient for a filter coefficient of the filter coefficients; decoding, from the compressed bitstream, a coefficient refinement value; and adjusting the predicted filter coefficient using the coefficient refinement value to obtain the filter coefficient.

[0143] In another example, a 7-tap convolutional filter may be used to obtain the chroma prediction block from a luminance prediction block. The convolutional filter may include a 5- tap plus sign shape spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of a center (C) luma sample which is collocated with the chroma sample to be predicted and its above/north (N), below/south (S), left/west (W) and right/east (E) neighbors, as described above with respect to equation (1). As such, the prediction pixel can be obtained using equation (2): predChromaVal = c₀C + c₄N + c₂S + c₃E + c₄W + c₅P + c₆B (2) [0144] In equation (2), P is the non-linear term. In an example, the nonlinear term P can be represented as power of two of the center luma sample C (i.e., intermediate pixel 1202) and scaled to the sample value range of the content using equation (3).

P = (C² + midVal » bitDepth. (3)

[0145] To illustrate, assuming 10-bit content, then P is calculated as P = ( C² + 512 ) » 10. Other non-linear terms are possible. The bias term B, when used, can represent a scalar offset between the input and output. The coefficients Ci can be obtained in a similar way as described above with respect to equation (1). For example, the coefficients Ci can be obtained by minimizing MSE between predicted and reconstructed chroma samples in a reference area. In equation (3), C, N, S, E, and W correspond to the values of the luma prediction values, such as shown in FIG. 12.

[0146] In an example, the coefficient refinement value corresponds (i.e., is used for) the intermediate prediction pixel itself. That is, for example, the coefficient refinement value may be used to refine the filter coefficient obtained for the intermediate pixel 1202 of FIG. 12. As such, the coefficient refinement value is used for an intermediate prediction pixel to which the filter is applied. In an example, the coefficient refinement value can be used to refine a coefficient corresponding to a non-linear term of the filter. For example, the filter may include a filter coefficient corresponding to the intermediate prediction pixel, one non-linear term, and a constant value. The non-linear term (i.e., a non-linear component) can be a square term of the intermediate prediction pixel. As such, the filter can be given by a X p[x]² + b X p[x] + c, where a and b are the filter coefficients, c is a constant component, and p[x] is the value of the intermediate prediction pixel at location x.

[0147] In an example, and as described with respect to FIG. 12, the filter coefficients can be applied to at least a subset of pixels in a 3x3 neighborhood of an intermediate prediction pixel to obtain the prediction pixel of the final prediction block. A 3x3 neighborhood can be used whether the current block is a luma block or a chroma block. The subset of the pixels can form (e.g., can be of any) shape. In an example, the subset of the pixels in a 3x3 neighborhood can be those pixels that form a cross shape, such as shown in FIG. 12. That is the subset of the pixels can be the pixels 1202 - 1210. That is, the at least a subset of pixels in a 3x3 neighborhood of an intermediate prediction pixel can be or include the intermediate prediction pixel, a pixel above the intermediate prediction pixel, a pixel right of the intermediate prediction pixel, a pixel below the intermediate prediction pixel, and a pixel left of the intermediate prediction pixel.

[0148] In an example, the filter can use at least a subset of pixels in a 3x3 neighborhood and may further include a constant term (also referred to as a DC value). In an example, one shape (i.e., a first filter shape) may be used for the at least the subset of pixels in the neighborhood for a current block that is a luma block and a second filter shape may be used for a current block that is a chroma block. As such, a first filter shape may be used in a case that the current block is a luma block and a second filter shape that is different from the first filter shape may be used in a case that the current block is a chroma block. [0149] FIG. 13 illustrates an example of obtaining a cross -component prediction. In cross-component filtering, a prediction obtained for a luma block can be used to obtain the prediction for a chroma block. Said another way, in the case that the current block is a luma block, a chroma prediction block for a chroma block corresponding to the current block is derived from the final prediction block. In an example, a 3x3 luma filter plus 1x1 chroma filter plus a DC value may be used. Alternatively, a 3x3 luma filter plus 3x3 chroma filter plus a DC value may be used.

[0150] Cross -component filtering can be similar to the cross-component filtering described in U.S. Patent Publication No. 2022/0272351, which is incorporated herein by reference. To summarize, in cross-component filtering, chroma samples are predicted based on the reconstructed luma samples of the same coding unit (which may be referred to as a largest coring unit, a macroblock, or other such nomenclature) of the current block by using a linear model that is according to equation (4): pred_c(i,j) = a x rec^' i,]) + /3 (4)

[0151] In equation (4), pred_c(i,f) represents the chroma sample predictions, rec_l' i,j') represents a down-sampled reconstructed luma predictions of the current luma block. Downsampling is performed in the case that the chroma samples and the luma samples do not have the same resolution. For example, down-sampling may be performed in that case that a 4:2:2 or a 4:2:0 format is used. The down-sampling aligns the resolution of luma and chroma blocks.

[0152] The cross-component parameters (a and >) can be derived with at most four neighboring chroma samples and their corresponding down-sampled luma samples. FIG. 13 illustrates an example 1300 of the locations of left and above samples and the sample of the current block involved in the cross-component filtering mode. The division operation to calculate parameter a may be implemented with a look-up table. With respect to a luma block 1302, the locations of left and above samples are shown as filled circles, such as a filled circle 1304. With respect to a chroma block 1306, the locations of left and above samples are shown as filled circles, such as a filled circle 1308.

[0153] FIG. 14 illustrates an example 1400 of obtaining a cross-component prediction. Obtaining the cross-component prediction may be referred to as a cross-component residual prediction technique. A cross -component residual prediction can be used in the case that a block is inter predicted or is predicted IBC and when there are non-zero luma residuals. The cross-component filters can be derived using the prediction signals of the luma block and the chroma block. The derived filters can be applied to the reconstructed luma signal producing a final chroma prediction.

[0154] The cross-component filters can be derived using the prediction signals of luma and chroma. The derived filters can be applied to the reconstructed luma signal producing the final chroma predictions. The filtering can use an 8-tap filter. The 8-tap filter can consist of 6 spatial luma samples, a nonlinear term, and a bias term. The spatial luma samples (L_o,...,L₅) can be obtained from the luma grid selecting the 6 luma samples closest to the chroma position C without down sampling. For example, given, a chroma pixel 1402, the luma values of the luma pixels at locations 1404-1414 can be used. The predicted chroma value can be obtained using equation (5): nonlinear( L₀ + L₃ + 1) » 1) + c₇B (5)

[0155] In equation (5), c₀ to c₇ are filter coefficients, nonlinear can be an operator that is as described with respect to equation (3), and B is a bias term.

[0156] For simplicity of explanation, the technique 900 of FIG. 9, is depicted and described as a series of steps or operations. However, the steps or operations in accordance with this disclosure can occur in various orders and/or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a technique in accordance with the disclosed subject matter.

[0157] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

[0158] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise or clearly indicated otherwise by the context, the statement “X includes A or B” is intended to mean any of the natural inclusive permutations thereof. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more,” unless specified otherwise or clearly indicated by the context to be directed to a singular form. Moreover, use of the term “an implementation” or the term “one implementation” throughout this disclosure is not intended to mean the same embodiment or implementation unless described as such.

[0159] Implementations of the transmitting station 102 and/or the receiving station 106 (and the algorithms, methods, instructions, etc., stored thereon and/or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0160] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general-purpose computer or general-purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms, and/or instructions described herein. In addition, or alternatively, for example, a special purpose computer/processor can be utilized which can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein. [0161] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device separate from the server, such as a handheld communications device. In this instance, the transmitting station 102, using an encoder 400, can encode content into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communications device, and/or a device including an encoder 400 may also include a decoder 500.

[0162] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport the program for use by or in connection with any processor. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0163] The above-described embodiments, implementations, and aspects have been described to facilitate easy understanding of this disclosure and do not limit this disclosure. On the contrary, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation as is permitted under the law to encompass all such modifications and equivalent arrangements.

Claims

What is claimed is:

1. A method, comprising: receiving a compressed bitstream comprising, with respect to a group of coding tree units, a luma plane followed by a chroma plane, wherein the luma plane constitutes data relating to luma pixel values and the chroma plane constitutes data relating to chroma pixel values; decoding a luma block of the luma plane based on a luma reference block included in another luma plane; and decoding a chroma block of the chroma plane based on a reference block, wherein the chroma block is collocated with the luma block.

2. The method of claim 1, wherein the reference block is decoded from the luma plane.

3. The method of claim 1, further comprising: decoding the reference block from the luma plane; and obtaining a post-filtered reference block from the reference block, wherein decoding the chroma block of the chroma plane based on the reference block comprises: decoding the chroma block based on the post-filtered reference block.

4. The method of any one of claims 1 to 3, wherein the reference block is decoded from the chroma plane.

5. The method of any one of claims 1 to 3, wherein the group of coding tree units is a frame.

6. The method of any one of claims 1 to 3, wherein the group of coding tree units constitutes a tile of a frame, wherein the frame is composed of multiple tiles.

7. The method of any one of claims 1 to 3, wherein the group of coding tree units constitutes a segment of a frame, wherein the frame is composed of multiple segments.

8. The method of any one of claims 1 to 3, wherein the group of coding tree units constitutes a slice of a frame, wherein the frame is composed of multiple slices.

9. The method of any one of claims 1 to 8, wherein the compressed bitstream further includes a coding information plane with respect to the group of coding tree units.

10. The method of claim 9, wherein the coding information plane is included in the luma plane.

11. The method of any one of claims 1 to 9, wherein the chroma plane is a first chroma plane, the compressed bitstream further comprising, with respect to a group of coding tree units (CTUs), a second chroma plane following the first chroma plane.

12. The method of any one of claims 1 to 11, further comprising: decoding, from the luma plane, a first tree corresponding to a first partitioning of blocks of the luma plane; and decoding, from the chroma plane, a second tree corresponding to a second partitioning of blocks of the chroma plane.

13. The method of claim 12, wherein a first structure of the first tree is different from a second structure of the second tree.

14. The method of claim 12, wherein the first tree is scanned using a first scanning order and the second tree is scanned using a second scanning order that is different from the first scanning order.

15. The method of any one of claims 1 to 14, wherein a first scanning order is used to scan coding tree units of the luma plane and a second scanning order that is different from the first scanning order is used to scan coding tree units of the chroma plane.

16. The method of any one of claims 1 to 15, wherein decoding the chroma block of the chroma plane based on the reference block comprises: generating a list of merge candidates; and selecting the reference block based on index into the list of the merge candidates decoded from the compressed bitstream.

17. The method of claim 16, wherein the list of merge candidates comprises temporal prediction candidates, wherein each of the temporal prediction candidates is based on a respective motion vector of a sub-block of the luma block of the luma plane.

18. The method of claim 17, further comprising: obtaining an intermediate prediction block corresponding to one of the temporal prediction candidates; and applying a filter to the intermediate prediction block to obtain a final prediction block.

19. The method of claim 17, further comprising: obtaining a first intermediate prediction block corresponding to one of the temporal prediction candidates; obtaining a second intermediate prediction block based on a cross-component prediction from the luma block; and combining the first intermediate prediction block and the second intermediate prediction block to obtain a final prediction block.

20. The method of claim 17, wherein the list of merge candidates comprises at least one candidate that is based on a prediction of the chroma block obtained from the luma block.

21. The method of claim 17, wherein the list of merge candidates comprises at least one candidate that is based on an intra block copy mode.

22. The method of claim 21, further comprising: filtering the at least one candidate based on filter coefficients obtained via a crosscomponent filtering operation.

23. The method of claim 16, wherein the list of merge candidates comprises a prediction candidate obtained using an intra-prediction mode.

24. A device, comprising: a processor that is configured to perform the method of any of claims 1-23.

25. A device, comprising: a memory; and a processor, the processor configured to execute instructions stored in the memory to perform the method of any of claims 1-23.

26. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising operations that perform the method of any of claims 1-23.

27. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is configured for decoding by the method of any of claims 1-23.