JP7482536B2 - Shape-adaptive discrete cosine transform for geometric partitioning with an adaptive number of regions. - Google Patents

Description

This application claims the benefit of priority to U.S. Provisional Patent Application No. 62/797,799, filed January 28, 2019, and entitled "SHAPE ADAPTIVE DISCRETE COSINE TRANSFORM FOR GEOMETRIC PARTITIONING WITH AN ADAPTIVE NUMBER OF REGIONS," which is incorporated herein by reference in its entirety.

The present invention relates generally to the field of video compression. In particular, the present invention is directed to a shape-adaptive discrete cosine transform for geometric partitioning with an adaptive number of regions.

A video codec may include electronic circuitry or software that compresses or decompresses digital video. It can convert uncompressed video to a compressed format, or vice versa. In the context of video compression, a device that compresses (and/or performs some of the functions of) the video may typically be called an encoder, and a device that decompresses (and/or performs some of the functions of) the video may be called a decoder.

The format of the compressed data can conform to standard video compression specifications. The compression can be lossy, in that the compressed video lacks certain information present in the original video. Consequences of this can include that the decompressed video may have lower quality than the original uncompressed video, because insufficient information exists to exactly reconstruct the original video.

There can be a complex relationship between video quality, the amount of data used to represent the video (e.g., determined by the bit rate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end delay (e.g., latency), and the like.

In one aspect, the decoder includes a circuit configured to receive a bitstream, determine a first region, a second region, and a third region of a current block according to a geometric partitioning mode, and decode the current block using an inverse discrete cosine transform for each of the first region, the second region, and the third region.

In another aspect, a decoder includes a circuit configured to receive a bitstream; determine a first region, a second region, and a third region of a current block according to a geometric partitioning mode; determine a coding transform law type for decoding each of the first region, the second region, and/or the third region from a signal contained in the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform and an inverse shape adaptive discrete cosine transform; and decode the current block, the decoding of the current block including using the determined transform law type for the inverse transform for each of the first region, the second region, and/or the third region.

In another aspect, a method includes a decoder receiving a bitstream; determining a first region, a second region, and a third region of a current block according to a geometric partitioning mode; determining a coding transform law type for decoding the first region, the second region, and/or the third region from a signal contained in the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform or an inverse shape adaptive discrete cosine transform; and decoding the current block, the decoding of the current block including using the determined transform law type for the inverse transform for each of the first region, the second region, and/or the third region.

The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims.
The present invention provides, for example, the following items.
(Item 1)
10. A decoder, the decoder comprising a circuit, the circuit comprising:
Receiving a bitstream;
determining a first region, a second region, and a third region of the current block according to a geometric partitioning mode;
decoding the current block using an inverse discrete cosine transform for each of the first region, the second region, and the third region;
A decoder configured to:
(Item 2)
2. The decoder of claim 1, wherein the current block has a size of 128x128 or 64x64.
(Item 3)
2. The decoder of claim 1, wherein a number of coefficients for an inverse transform of the first region, the second region, and/or the third region are signaled in the bitstream.
(Item 4)
determining whether the geometric partitioning mode is enabled; and
determining a first line segment for the current block;
determining a second line segment for the current block;
[0023] 20. The method according to claim 1, further comprising:
decoding the current block includes reconstructing pixel data using the first line segment and the second line segment;
2. The decoder of claim 1, wherein the first line segment and the second line segment divide the current block into the first region, the second region, and the third region.
(Item 5)
5. The decoder of claim 4, wherein the first line segment characterizes the first region and the second line segment characterizes the second region and the third region.
(Item 6)
5. The decoder of claim 4, wherein reconstructing pixel data includes calculating a predictor for the first region using an associated motion vector contained in the bitstream.
(Item 7)
an entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients;
an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine transformation, in accordance with the determined coding transform law type;
A deblocking filter;
A frame buffer;
Intra prediction processor
2. The decoder of claim 1, further comprising:
(Item 8)
2. The decoder of claim 1, wherein the bitstream includes a parameter indicating whether a geometric partitioning mode is enabled for the current block.
(Item 9)
2. The decoder of claim 1, wherein the current block forms part of a quad tree plus a binary decision tree.
(Item 10)
2. The decoder of claim 1, wherein the current block is a non-leaf node of the quadtree plus binary decision tree.
(Item 11)
2. The decoder of claim 1, wherein the current block is a coding tree unit or a coding unit.
(Item 12)
2. The decoder of claim 1, wherein the first region is a coding unit or a prediction unit.
(Item 13)
10. A decoder, the decoder comprising a circuit, the circuit comprising:
Receiving a bitstream;
determining a first region, a second region, and a third region of the current block according to a geometric partitioning mode;
determining a coding transform law type for decoding each of the first region, the second region, and/or the third region from a signal contained within the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform and an inverse shape adaptive discrete cosine transform;
decoding the current block, wherein decoding the current block includes using the determined transformation rule type for an inverse transformation for each of the first region, the second region, and/or the third region;
A decoder configured to:
(Item 14)
14. The decoder of claim 13, wherein the current block has a size of 128x128 or 64x64.
(Item 15)
14. The decoder of claim 13, wherein a number of coefficients for an inverse transform of the first region, the second region, and/or the third region are signaled in the bitstream.
(Item 16)
determining whether the geometric partitioning mode is enabled; and
determining a first line segment for the current block;
determining a second line segment for the current block;
[0023] 20. The method according to claim 1, further comprising:
decoding the current block includes reconstructing pixel data using the first line segment and the second line segment;
14. The decoder of claim 13, wherein the first line segment and the second line segment divide the current block into the first region, the second region, and the third region.
(Item 17)
17. The decoder of claim 16, wherein the first line segment characterizes the first region and the second line segment characterizes the second region and the third region.
(Item 18)
17. The decoder of claim 16, wherein reconstructing pixel data includes calculating a predictor for the first region using an associated motion vector contained in the bitstream.
(Item 19)
an entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients;
an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine in accordance with the determined coding transform law type;
A deblocking filter;
A frame buffer;
Intra prediction processor
Item 14. The decoder of item 13, further comprising:
(Item 20)
14. The decoder of claim 13, wherein the bitstream includes a parameter indicating whether a geometric partitioning mode is enabled for the current block.
(Item 21)
14. The decoder of claim 13, wherein the current block forms part of a quad tree plus a binary decision tree.
(Item 22)
14. The decoder of claim 13, wherein the current block is a non-leaf node of the quad tree plus binary decision tree.
(Item 23)
Item 14. The decoder of item 13, wherein the current block is a coding tree unit or a coding unit.
(Item 24)
Item 14. The decoder of item 13, wherein the first region is a coding unit or a prediction unit.
(Item 25)
1. A method, comprising:
receiving, by a decoder, a bitstream;
determining a first region, a second region, and a third region of the current block according to a geometric partitioning mode;
determining a coding transform law type for decoding the first region, the second region, and/or the third region from a signal contained in the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform or an inverse shape adaptive discrete cosine transform;
decoding the current block, wherein decoding the current block includes using the determined transformation rule type for an inverse transformation for each of the first region, the second region, and/or the third region;
A method comprising:
(Item 26)
26. The method of claim 25, wherein the current block has a size of 128x128 or 64x64.
(Item 27)
26. The method of claim 25, wherein a number of coefficients for an inverse transform of the first region, the second region, and/or the third region are signaled in the bitstream.
(Item 28)
determining, by the decoder, whether the geometric partitioning mode is enabled;
the decoder determining a first line segment for the current block;
the decoder determining a second line segment for the current block;
Further comprising:
decoding the current block includes reconstructing pixel data using the first line segment and the second line segment;
26. The method of claim 25, wherein the first line segment and the second line segment divide the current block into the first region, the second region, and the third region.
(Item 29)
29. The method of claim 28, wherein the first line segment characterizes the first region and the second line segment characterizes the second region and the third region.
(Item 30)
30. The method of claim 28, wherein reconstructing pixel data includes calculating a predictor for the first region using an associated motion vector contained in the bitstream.
(Item 31)
The decoder comprises:
an entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients;
an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine in accordance with the determined coding transform law type;
A deblocking filter;
A frame buffer;
Intra prediction processor
26. The method of claim 25, comprising:
(Item 32)
26. The method of claim 25, wherein the bitstream includes a parameter indicating whether a block-level geometric partitioning mode is enabled for the block.
(Item 33)
26. The method of claim 25, wherein the current block forms part of a quad tree plus a binary decision tree.
(Item 34)
26. The method of claim 25, wherein the current block is a non-leaf node of the quad tree plus binary decision tree.
(Item 35)
26. The method of claim 25, wherein the current block is a coding tree unit or a coding unit.
(Item 36)
26. The method of claim 25, wherein the first region is a coding unit or a prediction unit.

For the purpose of illustrating the invention, the drawings show aspects of one or more embodiments of the invention. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown in the drawings.

FIG. 1 is an illustration showing an example of a residual block (eg, a current block) with exponential partitioning, where there are three segments with different prediction errors.

FIG. 2 is a system block diagram illustrating an example video encoder capable of shape-adaptive discrete cosine transform (SA-DCT) for geometric partitioning with an adaptive number of regions, which can improve complexity and processing performance for video encoding and decoding.

FIG. 3 is a process flow diagram illustrating an exemplary process for encoding video using SA-DCT for geometric partitioning with an adaptive number of regions.

FIG. 4 is a system block diagram illustrating an example decoder capable of decoding a bitstream using SA-DCT for a geometric partition with an adaptive number of regions.

FIG. 5 is a process flow diagram illustrating one example process for decoding a bitstream using SA-DCT for a geometric partition with an adaptive number of regions.

FIG. 6 is a block diagram of a computing system that may be used to implement any one or more of the methods, or any one or more portions thereof, described herein.

The drawings are not necessarily to scale and may be illustrated by phantom lines, schematic representations, and partial views. In some instances, details that are not necessary for an understanding of the embodiments or that make other details difficult to perceive may be omitted. Like reference symbols in the various drawings indicate like elements.

The embodiments presented in this disclosure relate to encoding and decoding blocks in a geometric partitioning where not all blocks are necessarily rectangular. The embodiments may include and/or be configured to perform the encoding and/or decoding using a discrete cosine transform (DCT) and/or an inverse DCT. In some embodiments presented herein, the choice of DCT is made as a function of the information content within the geometrically partitioned block. In some existing video encoding and decoding approaches, all blocks are rectangular and the residual is encoded using a regular block DCT (B-DCT) on the entire rectangular block. However, in a geometric partitioning where a block may be partitioned into multiple non-rectangular regions, the use of a regular B-DCT may represent the underlying pixel information for some blocks inefficiently and may require unnecessary computational resources to implement. In some implementations of the present subject matter, when using a geometric partitioning mode, the encoder may use a shape-adaptive DCT (SA-DCT) as an alternative to or in addition to the B-DCT. In some embodiments, the encoder may select between B-DCT and SA-DCT for each region of a block, such as a geometrically partitioned block, based on the level of prediction error for that region, and the selection may be signaled in the bitstream for use in decoding. By encoding and/or decoding a non-rectangular region using either B-DCT or SA-DCT and signaling such a selection, the bit rate of transmission in the bitstream may be reduced, since the residual may be represented more efficiently, and the computational resources required to perform the processing may be reduced as a result. The present subject matter may be applicable to relatively large blocks, such as blocks having a size of 128x128 or 64x64. In some implementations, the geometric partitioning may involve dividing the current block into an adaptive number of regions (such as three or more regions for a given current block), and the DCT transform type (e.g., B-DCT or SA-DCT) may be signaled for each region.

In an embodiment, the B-DCT may be a DCT performed using an N×N invertible matrix on an N×N block of values, such as, but not limited to, the chroma and/or luma values of a corresponding N×N array of pixels. For example, as a non-limiting example, if an N×N matrix X is to be transformed, a "DCT-I" transform may calculate each element of the matrix to be transformed as follows:

where k=0,...,N-1. As a further non-limiting example, a "DCT-II" transform may calculate transformed matrix values as follows:

where k=0,...,N-1. As an illustrative example, if the block is a 4x4 pixel block, the generalized discrete cosine transform matrix may include a generalized discrete cosine transform II matrix taking the form:

In the formula, a is 1/2 and b is

and c is

It is.

In some implementations, integer approximations of the transform matrix may be utilized that may be used for efficient hardware and software implementations. For example, if the block is a 4×4 pixel block, the generalized discrete cosine transform matrix may include a generalized discrete cosine transform II matrix of the form:

The inverse B-DCT may be computed by a second matrix multiplication using the same N×N transform matrix, and the resulting output may be normalized to restore the original values. For example, the inverse DCT-I may be

It can be multiplied by

The SA-DCT may be performed on non-rectangular arrays of pixels. In an embodiment, the SA-DCT may be calculated by performing a one-dimensional version of the DCT, such as DCT-I, DCT-II, or equivalent, on vectors representing vertical columns of pixel values in the shape of interest, followed by grouping the resulting values into horizontal vectors and subjecting them to a second one-dimensional DCT, which may result in a completed transformation of the pixel values. Variations of the SA-DCT may further scale and/or normalize by a factor to correct mean-weighted and/or non-orthonormal defects introduced by the above transformation, quantization of the output of the above transformation, and/or inversion of the transformed output and/or the quantized transformed output. Further correction may be performed by subtracting the individual average value of the target image region from each pixel value, or a scaled version thereof, prior to the SA-DCT process described above, potentially in combination with one or other of the scaling processes applied before and/or after the transformation, quantization, and/or inverse transformation. Those skilled in the art will recognize, upon review of this disclosure in its entirety, various alternative or additional modifications to the SA-DCT process that may be applied consistently with the above description.

Motion compensation may include an approach for predicting a video frame or a portion thereof given previous and/or future frames by considering the motion of the camera and/or objects in the video including and/or represented by the current, previous, and/or future frames. Motion compensation may be employed in encoding and decoding video data for video compression, for example, in encoding and decoding using the Moving Picture Experts Group (MPEG)-2 (also referred to as Advanced Video Coding (AVC)) standard. Motion compensation may describe a picture in terms of the transformation of a reference picture into the current picture. The reference picture may be from a time earlier or future as compared to the current picture. Compression efficiency can be improved when pictures can be accurately synthesized from previously transmitted and/or stored pictures.

Block partitioning as used in this disclosure may refer to a method in video coding for finding regions of similar motion. Some forms of block partitioning can be found in video codec standards, including MPEG-2, H.264 (also referred to as AVC or MPEG-4 Part 10), and H.265 (also referred to as High Efficiency Video Coding (HEVC)). In an exemplary block partitioning approach, non-overlapping blocks of a video frame may be divided into rectangular sub-blocks to find block partitions containing pixels with similar motion. This approach may work well when all pixels of the block partition have similar motion. The motion of pixels within a block may be determined relative to a previously coded frame.

Shape-adaptive DCT and/or B-DCT may be effectively used in a geometric partitioning with an adaptive number of regions. Although FIG. 1 is an illustration showing a non-limiting example of a residual block (e.g., a current block) 100 of size 64×64 or 128×128 with a geometric partitioning in which there are three segments S0, S1, and S2 with different prediction errors, three segments are illustrated in FIG. 1 for illustrative purposes, and alternatively or additionally, a greater or lesser number of segments may be employed. The current block may be geometrically partitioned according to two line segments (P1P2 and P3P4), which may divide the current block into three regions S0, S1, and S2. In this example, S0 may have a relatively high prediction error, while S1 and S2 may have a relatively low prediction error. For segment S0 (also referred to as a region), the encoder may select and use B-DCT for residual coding. For segments S1 and S2 with low prediction errors, the encoder may select and use SA-DCT. The selection of the residual encoding transform may be based on the prediction error (e.g., the size of the residual). Because the SA-DCT algorithm is relatively simple from a complexity standpoint and does not require as many operations as the B-DCT, utilizing SA-DCT for residual coding with lower prediction errors may improve the complexity and processing performance for video encoding and decoding.

Thus, with continued reference to FIG. 1, SA-DCT may be signaled as an additional transform option to the full block DCT for segments with low prediction error. What is considered low error or high error may be a parameter that can be set at the encoder and can vary based on the application. The choice of transform type may be signaled in the bitstream. At the decoder, the bitstream may be analyzed and, for a given current block, the residual may be decoded using the transform type signaled in the bitstream. Alternatively or additionally, in some implementations, the number of coefficients associated with the transform may be signaled in the bitstream.

More specifically, and continuing to refer to FIG. 1, geometric partitioning with an adaptive number of regions may include techniques for video encoding and decoding in which a rectangular block is further divided into two or more regions that may be non-rectangular. For example, FIG. 1 illustrates a non-limiting example of geometric partitioning at the pixel level with an adaptive number of regions. An example rectangular block 100 (having a width of M pixels and a height of N pixels and may be represented as M×N pixels) may be divided into three regions (S0, S1, and S2) along line segments P1P2 and P3P4. When pixels in S0 have similar motion, a motion vector may describe the motion of all pixels in that region, and the motion vector may be used to compress region S0. Similarly, when pixels in region S1 have similar motion, an associated motion vector may describe the motion of pixels in region S1. Similarly, when pixels in region S2 have similar motion, an associated motion vector may describe the motion of pixels in region S2. Such a geometric division may be signaled to a receiver (e.g., a decoder) by encoding the positions P1, P2, P3, P4, and/or a representation of these positions using, for example, but not limited to, coordinates such as polar coordinates, Cartesian coordinates, or the like, an index into a predefined template, or other characterization of the division within the video bitstream.

Continuing with reference to FIG. 1, when encoding video data utilizing geometric partitioning at the pixel level, a line segment P1P2 (or, more specifically, points P1 and P2) may be determined. To determine the line segment P1P2 (or, more specifically, points P1 and P2) that best divides a block when utilizing geometric partitioning at the pixel level, the possible combinations of points P1 and P2 depend on M and N, which are the block width and height. For a block of size M×N, there are (M−1)×(N−1)×3 possible partitions. Thus, identifying the correct partition may be a computationally expensive task of evaluating motion estimates for all possible partitions, which may increase the amount of time and/or processing power required to encode the video compared to encoding using rectangular partitioning (e.g., without geometric partitioning at the pixel level). What constitutes the best or correct partition may be determined according to a metric and may vary from implementation to implementation.

In some implementations, with continued reference to FIG. 1, a first division may be determined that forms two regions (e.g., determining lines P1P2 and associated regions), and then the division is performed iteratively, in that one of the regions is further divided. For example, the division described with reference to FIG. 1 may be performed to divide a block into two regions. One of the regions may be further divided (e.g., to form new regions S1 and S2). This process may continue to perform block-level geometric divisions until a stopping criterion is reached.

2 is a system block diagram illustrating an exemplary video encoder 200 capable of SA-DCT and/or B-DCT for geometric partitioning with an adaptive number of regions, which can improve the complexity and processing performance for video encoding and decoding. The exemplary video encoder 200 receives an input video 205, which can first be segmented or divided according to a processing scheme such as a tree-structured macroblock partitioning scheme (e.g., a quad tree plus a binary tree). An example of a tree-structured macroblock partitioning scheme may include partitioning a picture frame into large block elements called coding tree units (CTUs). In some implementations, each CTU may be further partitioned one or more times into several sub-blocks called coding units (CUs). The end result of this partitioning may include a group of sub-blocks that may be called prediction units (PUs). Transform units (TUs) may also be utilized. Such partitioning schemes may include implementing a geometric partitioning with an adaptive number of regions in accordance with some aspects of the present subject matter.

Continuing with reference to FIG. 2, the exemplary video encoder 200 includes an intra-prediction processor 215, a motion estimation/compensation processor 220 (also referred to as an inter-prediction processor) capable of supporting geometric partitioning with an adaptive number of regions, a transform/quantization processor 225, an inverse quantization/inverse transform processor 230, an in-loop filter 235, a decoded picture buffer 240, and an entropy coding processor 245. In some implementations, the motion estimation/compensation processor 220 can perform the geometric partitioning. A bitstream parameter signaling the geometric partitioning mode can be input to the entropy coding processor 245 for inclusion in the output bitstream 250.

In operation, and continuing to refer to FIG. 2, for each block of a frame of the input video 205, a decision can be made whether to process the block via intra-picture prediction or using motion estimation/compensation. The block can be provided to an intra-prediction processor 210 or a motion estimation/compensation processor 220. If the block is to be processed via intra-prediction, the intra-prediction processor 210 can perform processing and output a predictor. If the block is to be processed via motion estimation/compensation, the motion estimation/compensation processor 220 can perform processing, including the use of geometric partitioning, and output a predictor.

Continuing to refer to FIG. 2, a residual may be formed by subtracting a predictor from the input video. The residual may be received by a transform/quantization processor 225, which may determine (e.g., by comparing the size of the residual or an error metric to a threshold) whether the prediction error (e.g., residual size) is considered to be a "high" error or a "low" error. Based on the determination, the transform/quantization processor 225 may select a transform type, which may include B-DCT and SA-DCT. In some implementations, the transform/quantization processor 225 selects a transform type of B-DCT if the residual is considered to have a high error and selects a transform type of SA-DCT if the residual is considered to have a low error. Based on the selected transform type, the transform/quantization processor 225 may perform a transform process (e.g., SA-DCT or B-DCT) to generate coefficients, which may be quantized. The quantized coefficients and any associated signaling information (which may include the selected transform type and/or the number of coefficients used) may be provided to an entropy coding processor 245 for entropy encoding and inclusion in the output bitstream 250. The entropy encoding processor 245 may support encoding of signaling information related to the SA-DCT for geometric partitioning with an adaptive number of regions. In addition, the quantized coefficients may be provided to an inverse quantization/inverse transform processor 230, which may reconstruct pixels that may be combined with a predictor and processed by an in-loop filter 235, the output of which is stored in a decoded picture buffer 240 for use by a motion estimation/compensation processor 220, which may support geometric partitioning with an adaptive number of regions.

Now referring to FIG. 3, a process flow diagram illustrating an example process 300 for encoding video using SA-DCT for geometric partitioning with an adaptive number of regions, which can improve complexity and processing performance for video encoding and decoding, is illustrated. At step 310, a video frame may undergo initial block segmentation using, for example, a tree-structured macroblock partitioning scheme, which may include partitioning a picture frame into CTUs and CUs. At 320, a block may be selected for geometric partitioning. The selection may include identifying, according to a metric rule, that the block should be processed according to a geometric partitioning mode. At step 330, the selected block may be partitioned into three or more non-rectangular regions according to the geometric partitioning mode.

At step 340, and still referring to FIG. 3, for each geometrically divided region, a transform type (also referred to as a transform law type) may be determined. This may include determining whether the prediction error (e.g., residual size) is considered to be a "high" or "low" error (e.g., by comparing the size of the residual or an error metric to a threshold). Based on the determination, a transform type may be selected, for example, using a quad tree plus binary decision tree process as described below, which may include, but is not limited to, B-DCT or SA-DCT. In some implementations, if the residual is considered to have a high error, a B-DCT transform type is selected, and if the residual is considered to have a low error, a SA-DCT transform type is selected. Based on the selected transform type, a transform process (e.g., SA-DCT or B-DCT) may be performed to generate coefficients that may be quantized.

At step 350, and continuing to refer to FIG. 3, the determined transform type may be signaled in the bitstream. The transformed and quantized residual may be included in the bitstream. In some implementations, the number of transform coefficients may be signaled in the bitstream.

FIG. 4 is a system block diagram illustrating a non-limiting example of a decoder 400 capable of decoding a bitstream 470 using DCT including, but not limited to, SA-DCT and/or B-DCT for geometric partitioning with an adaptive number of regions, which can improve complexity and processing performance for video encoding and decoding. The decoder 400 includes an entropy decoder processor 410, an inverse quantization and inverse transform processor 420, a deblocking filter 430, a frame buffer 440, a motion compensation processor 450, and an intra prediction processor 460. In some implementations, the bitstream 470 includes parameters signaling a geometric partitioning mode and a transform law type. In some implementations, the bitstream 470 includes parameters signaling a number of transform coefficients. The motion compensation processor 450 can reconstruct pixel information using the geometric partitioning as described herein.

In operation, and still referring to FIG. 4, a bitstream 470 may be received by the decoder 400 and input to the entropy decoder processor 410, which may entropy decode the bitstream into quantized coefficients. The quantized coefficients may be provided to the inverse quantization and inverse transform processor 420, which may determine a coding transform law type (e.g., B-DCT or SA-DCT) and perform inverse quantization and inverse transform according to the determined coding transform law type to create a residual signal. In some implementations, the inverse quantization and inverse transform processor 420 may determine the number of transform coefficients and perform the inverse transform according to the determined number of transform coefficients.

Continuing to refer to FIG. 4, the residual signal may be added to the output of the motion compensation processor 450 or the intra-prediction processor 460, depending on the processing mode. The output of the motion compensation processor 450 and the intra-prediction processor 460 may include a block prediction based on a previously decoded block. The sum of the prediction and the residual may be processed by the deblocking filter 430 and stored in the frame buffer 440. For a given block (e.g., CU or PU), when the bitstream 470 signals that the partition mode is block-level geometric partitioning, the motion compensation processor 450 may construct a prediction based on the geometric partitioning approach described herein.

FIG. 5 is a process flow diagram illustrating an example process 500 for decoding a bitstream using SA-DCT for geometric partitioning with an adaptive number of regions, which can improve complexity and processing performance for video encoding and decoding. At step 510, a bitstream is received, which may include a current block (e.g., CTU, CU, PU). Receiving may include extracting and/or parsing the current block and associated signaling information from the bitstream. The decoder may extract or determine one or more parameters that characterize the geometric partitioning. These parameters may include, for example, indices of start and end points of line segments (e.g., P1, P2, P3, P4), and extracting or determining may include identifying and retrieving the parameters from the bitstream (e.g., parsing the bitstream).

At step 520, with continued reference to FIG. 5, a first region, a second region, and a third region of the current block may be determined according to a geometric partitioning mode. Determining may include determining whether the geometric partitioning mode is enabled (e.g., true) for the current block. If the geometric partitioning mode is not enabled (e.g., false), the decoder may process the current block using an alternate partitioning mode. If the geometric partitioning mode is enabled (e.g., true), more than two regions may be determined and/or processed.

At optional step 530, and continuing to refer to FIG. 5, a coding transform law type may be determined. The coding transform law type may be signaled within the bitstream. For example, the bitstream may be analyzed to determine a coding transform law type, which may specify B-DCT or SA-DCT. The determined coding transform law type may be for decoding the first region, the second region, and/or the third region.

At 540, with continued reference to FIG. 5, the current block may be decoded. Decoding the current block may include using the determined transform type for an inverse transform for each of the first region, the second region, and/or the third region. The decoding may include, for each region, determining associated motion information according to the geometric partitioning mode.

Although a few variations have been detailed above, other modifications or additions are possible. For example, the geometric partitioning can be signaled in the bitstream based on a rate-distortion decision at the encoder. The coding can be based on a combination of regular predefined partitioning (e.g., a template), temporal and spatial prediction of the partitioning, and an additive offset. Each of the geometrically partitioned regions can utilize motion compensated prediction or intra prediction. The boundaries of the predicted regions can be smoothed before the residual is added.

In some implementations, a quad-tree plus binary decision tree (QTBT) may be implemented, where at the coding tree unit level, the splitting parameters of the QTBT are dynamically derived to adapt to local characteristics without transmitting any overhead. Subsequently, at the coding unit level, a joint classifier decision tree structure may eliminate unnecessary iterations and control the risk of erroneous predictions. In some implementations, a geometric split with an adaptive number of regions may be available as an additional splitting option available at all leaf nodes of the QTBT.

In some implementations, the decoder may include a partitioning processor that generates a geometric partitioning for the current block and provides all partition-related information for the subordinate processes. The partitioning processor may directly affect motion compensation, since this may be performed on a segment-by-segment basis if the block is geometrically partitioned. Additionally, the partitioning processor may provide shape information to the intra-prediction processor and the transform coding processor.

In some implementations, additional syntax elements may be signaled at different hierarchical levels of the bitstream. An enable flag may be coded in the sequence parameter set (SPS) to enable geometric partitioning with an adaptive number of regions for the entire sequence. Additionally, a CTU flag may be coded at the coding tree unit (CTU) level to indicate whether a coding unit (CU) uses geometric partitioning with an adaptive number of regions. A CU flag may be coded to indicate whether the current coding unit utilizes geometric partitioning with an adaptive number of regions. Parameters that specify line segments on a block may be coded. For each region, a flag may be decoded that may specify whether the current region is inter-predicted or intra-predicted.

In some implementations, a minimum region size may be specified.

The subject matter described herein provides many technical advantages. For example, some implementations of the subject matter can provide block partitioning that reduces complexity while increasing compression efficiency. In some implementations, blocking artifacts at object boundaries can be reduced.

It should be noted that any one or more of the aspects and embodiments described herein may be conveniently implemented using digital electronic circuitry, integrated circuits, specially designed application specific integrated circuits (ASICs), field programmable gate array (FPGA) computer hardware, firmware, software, and/or combinations thereof realized and/or implemented in one or more machines (e.g., one or more computing devices utilized as user computing devices for electronic documents, one or more server devices such as document servers, etc.) programmed in accordance with the teachings herein, as would be apparent to one of ordinary skill in the computer arts. These various aspects or features may include implementation in one or more computer programs and/or software executable and/or readable on a programmable system including at least one programmable processor, which may be dedicated or general purpose, coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device. Appropriate software coding may be readily prepared by skilled programmers based on the teachings of the present disclosure, as would be apparent to one of ordinary skill in the software arts. The aspects and implementations discussed above that employ software and/or software modules may also include suitable hardware to assist in implementing the machine-executable instructions of the software and/or software modules.

Such software may be a computer program product employing a machine-readable storage medium. A machine-readable storage medium may be any medium capable of storing and/or encoding a sequence of instructions for execution by a machine (e.g., a computing device) and causing the machine to perform any one of the methods and/or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, magnetic disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state memory devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and/or any combination thereof. Machine-readable media, as used herein, is intended to include a single medium as well as a collection of physically separate media, such as, for example, a collection of compact discs or one or more hard disk drives in combination with a computer memory. As used herein, machine-readable storage media does not include a transitory form of signal transmission.

Such software may also include information (e.g., data) carried as a data signal on a data carrier, such as a carrier wave. For example, machine-executable information may be included as a data carrying signal embodied in a data carrier, the signal encoding a sequence of instructions or portions thereof for execution by a machine (e.g., a computing device), as well as any associated information (e.g., data structures and data) that causes the machine to perform any one of the methods and/or embodiments described herein.

Examples of computing devices include, but are not limited to, e-book reading devices, computer workstations, terminal computers, server computers, handheld devices (e.g., tablet computers, smartphones, etc.), web appliances, network routers, network switches, network bridges, any machine capable of executing a sequence of instructions that define actions to be taken by the machine, and any combination thereof. In one example, a computing device may include and/or be included within a kiosk.

6 illustrates a diagrammatic representation of one embodiment of a computing device as an exemplary form of computer system 600 on which a set of instructions for causing a control system to perform any one or more of the aspects and/or methods of the present disclosure may be executed. It is also contemplated that multiple computing devices may be utilized to implement a set of instructions specifically configured to cause one or more of the devices to perform any one or more of the aspects and/or methods of the present disclosure. Computer system 600 includes a processor 604 and a memory 608, which communicate with each other and with other components via a bus 612. Bus 612 may include any of several types of bus structures, including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combination thereof, using any of a variety of bus architectures.

The memory 608 may include a variety of components (e.g., machine-readable media), including, but not limited to, random access memory components, read-only components, and any combination thereof. In one example, a basic input/output system 616 (BIOS), including basic routines that help to transfer information between elements within the computer system 600, such as during startup, may be stored in the memory 608. The memory 608 may also include (e.g., stored on one or more machine-readable media) instructions (e.g., software) 620 that embody any one or more of the aspects and/or methods of the present disclosure. In another example, the memory 608 may further include any number of program modules, including, but not limited to, an operating system, one or more application programs, other program modules, program data, and any combination thereof.

Computer system 600 may also include a storage device 624. Examples of storage devices (e.g., storage device 624) include, but are not limited to, hard disk drives, magnetic disk drives, optical disk drives combined with optical media, solid-state memory devices, and any combination thereof. Storage device 624 may be connected to bus 612 by an appropriate interface (not shown). Exemplary interfaces include, but are not limited to, SCSI, Advanced Technology Attachment (ATA), Serial ATA, Universal Serial Bus (USB), IEEE 1394 (FIREWIRE®), and any combination thereof. In one example, storage device 624 (or one or more of its components) may be removably interfaced with computer system 600 (e.g., via an external port connector (not shown)). In particular, storage device 624 and associated machine-readable media 628 may provide non-volatile and/or volatile storage of machine-readable instructions, data structures, program modules, and/or other data for computer system 600. In one example, the software 620 may reside, completely or partially, within the machine-readable medium 628. In another example, the software 620 may reside, completely or partially, within the processor 604.

Computer system 600 may also include input devices 632. In one example, a user of computer system 600 may type commands and/or other information into computer system 600 via input devices 632. Examples of input devices 632 include, but are not limited to, alphanumeric input devices (e.g., keyboards), pointing devices, joysticks, gamepads, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touchpads, optical scanners, video capture devices (e.g., still cameras, video cameras), touch screens, and any combination thereof. Input devices 632 may be interfaced to bus 612 via any of a variety of interfaces (not shown), including, but not limited to, a serial interface, a parallel interface, a game port, a USB interface, a FIREWIRE® interface, a direct interface to bus 612, and any combination thereof. Input devices 632 may include a touch screen interface, which may be part of or separate from display 636, discussed further below. The input device 632 may be utilized as a user selection device for selecting one or more graphical representations in a graphical interface as described above.

A user may also input commands and/or other information to computer system 600 via storage device 624 (e.g., removable disk drive, flash drive, etc.) and/or network interface device 640. A network interface device, such as network interface device 640, may be utilized to connect computer system 600 to one or more of a variety of networks, such as network 644, and one or more remote devices 648 connected thereto. Examples of network interface devices include, but are not limited to, network interface cards (e.g., mobile network interface cards, LAN cards), modems, and any combination thereof. Examples of networks include, but are not limited to, wide area networks (e.g., the Internet, enterprise networks), local area networks (e.g., networks associated with an office, building, campus, or other relatively small geographic space), telephone networks, data networks associated with a telephone/voice provider (e.g., a mobile communications provider's data and/or voice network), a direct connection between two computing devices, and any combination thereof. A network, such as network 644, may employ wired and/or wireless modes of communication. In general, any network topology may be used. Information (e.g., data, software 620, etc.) may be communicated to and/or from computer system 600 via network interface device 640.

The computer system 600 may further include a video display adapter 652 for communicating images displayable on a display device, such as the display device 636. Examples of display devices include, but are not limited to, a liquid crystal display (LCD), a cathode ray tube (CRT), a plasma display, a light emitting diode (LED) display, and any combination thereof. The display adapter 652 and the display device 636 may be utilized in combination with the processor 604 to provide graphical representations of aspects of the present disclosure. In addition to a display device, the computer system 600 may include one or more other peripheral output devices, including, but not limited to, audio speakers, a printer, and any combination thereof. Such peripheral output devices may be connected to the bus 612 via a peripheral interface 656. Examples of peripheral interfaces include, but are not limited to, a serial port, a USB connection, a FIREWIRE® connection, a parallel connection, and any combination thereof.

The foregoing is a detailed description of illustrative embodiments of the present invention. Various modifications and additions may be made without departing from the spirit and scope of the present invention. Features of each of the various embodiments described above may be combined with features of other described embodiments, as appropriate, to provide a combination of features in related new embodiments. Moreover, while the foregoing describes several separate embodiments, what has been described herein is merely illustrative of the application of the principles of the present invention. In addition, although certain methods herein may be illustrated and/or described as being performed in a specific order, the order may be varied considerably within ordinary skill in the art to achieve the embodiments as disclosed herein. Thus, this description is intended to be taken only as an example, and is not intended to otherwise limit the scope of the present invention.

In the above description and in the claims, phrases such as "at least one of" or "one or more of" may occur followed by a conjunctive enumeration of elements or features. The term "and/or" may also occur within a enumeration of two or more elements or features. Unless otherwise implied or explicitly contradicted by the context in which such a phrase is used, this is intended to mean any of the elements or features listed individually or any of the elements or features listed in combination with any of the other listed elements or features. For example, the phrases "at least one of A and B," "one or more of A and B," and "A and/or B" are each intended to mean "A only, B only, or both A and B." A similar interpretation is intended with respect to enumerations containing more than two items. For example, the phrases "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, and/or C" are each intended to mean "A only, B only, C only, both A and B, both A and C, both B and C, or both A, B, and C." Additionally, use of the term "based on" above and in the claims is intended to mean "based at least on," such that unrecited features or elements are also allowed.

The subject matter described herein may be embodied as a system, an apparatus, a method, and/or an article, depending on the desired configuration. The implementations described in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the subject matter described. Although some variations have been described in detail above, other modifications or additions are possible. In particular, further features and/or variations may be provided in addition to those described herein. For example, the implementations described above may be directed to various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of some further features disclosed above. In addition, the logic flow depicted in the accompanying figures and/or described herein does not necessarily require the particular order or sequential order shown to achieve the desired results. Other implementations may be within the scope of the following claims.

Claims (30)

10. A decoder, the decoder comprising a circuit, the circuit comprising:
Receiving a bitstream;
determining a first region, a second region, and a third region of the current block according to a geometric partitioning mode;
decoding the current block using an inverse discrete cosine transform for each of the first region, the second region, and the third region;
determining whether the geometric partitioning mode is enabled; and
determining a first line segment for the current block;
determining a second line segment for the current block, wherein decoding the current block includes reconstructing pixel data using the first line segment and the second line segment;
The first line segment and the second line segment divide the current block into the first region, the second region, and the third region. The decoder of claim 1, wherein the current block has a size of 128x128 or 64x64. The decoder of claim 1, wherein the first line segment characterizes the first region and the second line segment characterizes the second region and the third region. The decoder of claim 1, wherein reconstructing pixel data includes calculating a predictor for the first region using an associated motion vector contained in the bitstream. The circuitry is further configured to determine a coding transform law type for decoding each of the first region, the second region, and/or the third region from a signal contained in the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform and an inverse shape adaptive discrete cosine transform, the decoder comprising:
an entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients;
an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine transformation, in accordance with the determined coding transform law type;
A deblocking filter;
A frame buffer;
The decoder of claim 1 further comprising: an intra-prediction processor; The decoder of claim 1, wherein the bitstream includes a parameter indicating whether the geometric partitioning mode is enabled for the current block. The decoder of claim 1, wherein the current block forms part of a quad tree plus a binary decision tree. The decoder of claim 1, wherein the current block is a non-leaf node of a quad tree plus a binary decision tree. The decoder of claim 1, wherein the current block is a coding tree unit or a coding unit. The decoder of claim 1, wherein the first region is a coding unit or a prediction unit. 10. A decoder, the decoder comprising a circuit, the circuit comprising:
Receiving a bitstream;
determining a first region, a second region, and a third region of the current block according to a geometric partitioning mode;
determining a coding transform law type for decoding each of the first region, the second region, and/or the third region from a signal contained within the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform and an inverse shape adaptive discrete cosine transform;
decoding the current block, wherein decoding the current block includes using the determined transformation rule type for an inverse transformation for each of the first region, the second region, and/or the third region;
determining whether the geometric partitioning mode is enabled; and
determining a first line segment for the current block;
determining a second line segment for the current block, wherein decoding the current block includes reconstructing pixel data using the first line segment and the second line segment;
The first line segment and the second line segment divide the current block into the first region, the second region, and the third region. The decoder of claim 11 , wherein the current block has a size of 128×128 or 64×64. 12. The decoder of claim 11 , wherein the first line segment characterizes the first region and the second line segment characterizes the second region and the third region. 12. The decoder of claim 11 , wherein reconstructing pixel data includes calculating a predictor for the first region using an associated motion vector contained in the bitstream. an entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients;
an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine in accordance with the determined coding transform law type;
A deblocking filter;
A frame buffer;
The decoder of claim 11 further comprising: an intra-prediction processor; The decoder of claim 11 , wherein the bitstream includes a parameter indicating whether the geometric partitioning mode is enabled for the current block. 12. The decoder of claim 11 , wherein the current block forms part of a quad-tree plus a binary decision tree. The decoder of claim 11 , wherein the current block is a non-leaf node of a quadtree plus a binary decision tree. The decoder of claim 11 , wherein the current block is a coding tree unit or a coding unit. The decoder of claim 11 , wherein the first region is a coding unit or a prediction unit. 1. A method, comprising:
A decoder receives a bitstream;
determining a first region, a second region, and a third region of the current block according to a geometric partitioning mode;
determining a coding transform law type for decoding the first region, the second region, and/or the third region from a signal contained in the bitstream, the coding transform law type characterizing at least an inverse block discrete cosine transform or an inverse shape adaptive discrete cosine transform;
decoding the current block, wherein decoding the current block includes using the determined transformation rule type for an inverse transformation for each of the first region, the second region, and/or the third region;
determining, by the decoder, whether the geometric partitioning mode is enabled;
the decoder determining a first line segment for the current block;
the decoder determining a second line segment for the current block;
decoding the current block includes reconstructing pixel data using the first line segment and the second line segment;
The method of claim 1, wherein the first line segment and the second line segment divide the current block into the first region, the second region, and the third region. The method of claim 21 , wherein the current block has a size of 128×128 or 64×64. 22. The method of claim 21 , wherein the first line segment characterizes the first region and the second line segment characterizes the second region and the third region. 22. The method of claim 21 , wherein reconstructing pixel data includes calculating a predictor for the first region using an associated motion vector contained in the bitstream. The decoder comprises:
an entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients;
an inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine in accordance with the determined coding transform law type;
A deblocking filter;
A frame buffer;
and an intra-prediction processor. 22. The method of claim 21 , wherein the bitstream includes a parameter indicating whether a block level geometric partitioning mode is enabled for the current block . 22. The method of claim 21 , wherein the current block forms part of a quad tree plus a binary decision tree. 22. The method of claim 21 , wherein the current block is a non-leaf node of a quad tree plus a binary decision tree. The method of claim 21 , wherein the current block is a coding tree unit or a coding unit. The method of claim 21 , wherein the first region is a coding unit or a prediction unit.