RU2829207C2 - Interframe prediction at exponential division - Google Patents

Abstract

FIELD: decoding means.

SUBSTANCE: receiving a bit stream including an encoded image, wherein the encoded image includes a block of encoded pixels and signaling information including a first division index used to determine an indirect non-rectangular division starting point, and a second separation index used to determine the end point of the indirect non-rectangular separation. Determining, using the first division index, the initial point of the indirect non-rectangular division in the block, wherein the starting point is located on the first side of the block with a first offset distance from the first corner of the block. Determining, using the second division index, the end point of the indirect non-rectangular division in the block, wherein the end point is located on the second side of the block with a second offset distance from the second corner of the block. First pixel prediction values are generated in the first region on the first side relative to the indirect non-rectangular partitioning. Second pixel prediction values are generated in the second region on the second side relative to the indirect non-rectangular partitioning. Block is decoded using pixel prediction values.

EFFECT: high efficiency of image compression.

8 cl, 18 dwg

Description

Field of technology to which the invention relates

The present invention relates generally to the field of video compression. In particular, the present invention is directed to interframe prediction with exponential partitioning.

State of the art

A video codec may include electronic circuitry or software that compresses or decompresses digital video. It may convert uncompressed video to a compressed format and vice versa. In the context of video compression, a device that compresses video (and/or performs some of its functions) may generally be called an encoder, and a device that decompresses video (and/or performs some of its functions) may be called a decoder.

The compressed data format may conform to a standard video compression specification. The compression may be lossy in the sense that the compressed video lacks some information present in the original video. The consequence of this may be that the decompressed video may be of lower quality than the original uncompressed video, since there is not enough information to accurately reconstruct the original video.

Complex relationships may exist between video quality, the amount of data used to represent the video (e.g., as determined by the bit rate), the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end delay (e.g., lag), etc.

Disclosure of the essence of the invention

In one aspect, a decoder includes a circuit configured to receive a bit stream, divide a current block using an exponential division mode into a first region and a second region, determine a motion vector associated with a region from the first region or the second region, where the determining includes generating a candidate list, and decoding the current block using the determined motion vector.

In another aspect, the method includes receiving, by a decoder, a bit stream, dividing, by the decoder, a current block using an exponential division mode into a first region and a second region, determining, by the decoder, a motion vector associated with a region of the first region or the second region, determining includes forming a candidate list, and decoding, by the decoder, the current block using the determined motion vector.

The details of one or more embodiments of the subject matter of the invention described herein are set forth below in the accompanying drawings and in the description. Other features and advantages of the subject matter of the invention described herein will be apparent from the description and drawings and from the claims.

Brief description of the drawings

For purposes of illustrating the invention, the drawings show aspects of one or more embodiments of the invention. However, it should be understood that the present invention is not limited to the precise arrangements and instruments shown in the drawings, in which:

Fig. 1 – diagram illustrating an example of block division of pixels;

Fig. 2 is a diagram illustrating an example of geometric separation;

Fig. 3 – an image containing an apple that cannot be effectively divided by curved segments;

Fig. 4A is a diagram illustrating an example of exponential partitioning according to certain aspects of the present subject matter that can improve compression efficiency;

Fig. 4B is a series of diagrams illustrating exemplary template exponential sections;

Fig. 4C - example curves associated with 4 given coefficients that can define an example exponential function;

Fig. 4D is another exemplary block showing the different starting indices P1 and ending indices P2 that divide the rectangular block;

Fig. 5 is a diagram illustrating exemplary positions of potential motion vector candidates relative to an exemplary current block partitioned according to exponential partitioning;

Fig. 6 illustrates Fig. 5 with an annotation showing luminance locations including the upper leftmost luminance location of the first region and the upper rightmost luminance location of the second region;

Fig. 7 is a diagram illustrating the positions of potential motion vector candidates relative to an exemplary current block divided according to exponential partitioning;

Fig. 8 illustrates Fig. 7 with an annotation showing luminance locations including the leftmost lower luminance location of the second region and the rightmost upper luminance location of the second region;

Fig. 9 is a block diagram of a system illustrating an example of a video encoder configured to encode video using interframe prediction with exponential division;

Fig. 10 – example of QTBT frame division;

Fig. 11 is an example of exponential splitting at the CU QTBT level shown in Fig. 8;

Fig. 12 is a flow chart illustrating an exemplary process for coding video with exponential partitioning and interframe prediction according to certain aspects of the present subject matter, which reduces coding complexity while increasing compression efficiency;

Fig. 13 is a block diagram of a system illustrating an example decoder configured to decode a bitstream using exponential division and inter-frame prediction according to certain aspects of the present subject matter;

Fig. 14 is a flow chart illustrating an example process for decoding a bitstream using exponential partitioning and inter-frame prediction according to certain aspects of the present subject matter; and

Fig. 15 is a block diagram of a computing system that can be used to implement any one or more of the methodologies disclosed herein and any one or more portions thereof.

The drawings are not necessarily drawn to scale and may be illustrated using dashed lines, schematic representations and fragmentary views. In some cases, details that are not necessary for understanding the embodiments or that obscure other details may be omitted. The same reference numerals in different drawings designate the same elements.

Implementation of the invention

Some implementations of the present subject matter include performing inter-picture prediction with non-rectangular regions that have been divided by a curve, wherein the curve may or may not be a straight line. Performing inter-picture prediction with non-rectangular blocks that have been divided by a curve allows for a division that will more accurately follow the object boundaries, which will result in a lower motion compensation prediction error, smaller residuals and, thus, an increase in compression efficiency. During inter-picture prediction, motion compensation can be performed using motion vectors predicted for blocks (e.g., coding units, prediction units, etc.) determined according to an exponential division mode. Motion vectors can be predicted using enhanced motion vector prediction (AMVP) and/or using a fusion mode, where the motion vector is selected from a list of motion vector candidates without encoding the difference of the motion vectors.

In exponential partitioning, a rectangular block may be partitioned into non-rectangular regions by a curve, which may include a straight line segment in the case of geometric partitioning, or may be, more generally, a curve that is not a straight line. Using a curve that is not a straight line to partition blocks allows for a partition that follows object boundaries more closely, resulting in lower motion compensation prediction error, smaller residuals, and thus improved compression efficiency. In some implementations, the curve may be characterized by an exponential function. The curve (e.g., the exponential function) may be defined using specified coefficients that may be signaled in the bitstream for use by the decoder. In some implementations, exponential partitioning may be available for luma samples greater than or equal to 8x8. By dividing rectangular blocks of curve, the current object allows for greater compression efficiency for certain objects than technologies limited to dividing into straight-line segments, such as using geometric partitioning.

Motion compensation may include an approach to predicting a video frame or part of a video frame given previous and/or future frames by taking into account the motion of the camera and/or objects in the video. Motion compensation may be used in encoding and decoding video data for video compression, such as encoding and decoding using the Moving Picture Experts Group (MPEG)-2 standard (also called Advanced Video Coding (AVC)). Motion compensation allows a picture to be described in terms of a transformation from a reference picture to a current picture. The reference picture may be a previous or future picture in time compared to the current picture. When pictures can be accurately synthesized from previously transmitted and/or stored pictures, compression efficiency can be improved.

Block partitioning can refer to a method of encoding video to find areas of similar motion. Some form of block partitioning can be found in video codec standards including MPEG-2, H.264 (also called AVC or MPEG-4 Part 10), and H.265 (also called High Efficiency Video Coding (HEVC)). In exemplary approaches to block partitioning, non-overlapping blocks of a video frame can be divided into rectangular sub-blocks to find block partitions that contain pixels with similar motion. This approach can work well when all the pixels in a block partition have similar motion. The motion of pixels in a block can be determined relative to previously encoded frames.

Fig. 1 shows a diagram illustrating an example of block division of pixels. An original rectangular picture or block 100, which itself can be a sub-block (e.g., a node in a coding tree), can be divided into rectangular sub-blocks. For example, in step 110, the block 100 is divided into two rectangular sub-blocks 110a and 110b. Sub-blocks 110a and 110b can then be processed separately. As another example, in step 120, the block 100 is divided into four rectangular sub-blocks 120a, 120b, 120c and 120d. The sub-blocks themselves can be further divided until it is determined that the pixels within the sub-blocks share the same motion, a minimum block size is reached, or other criteria are met. When the pixels in a sub-block have similar motion, the motion vector can describe the motion of all pixels in this area.

Still referring to Fig. 1, some approaches to video coding may include geometric partitioning, which may be a form of exponential partitioning, in which a rectangular block (e.g., as shown in Fig. 1) is further divided by a rectilinear segment into two regions, which may be non-rectangular. For example, Fig. 2 shows a diagram illustrating an example of geometric partitioning. An exemplary rectangular block 200 (which may have a width of M pixels and a height of N pixels, denoted as M×N pixels) may be divided along a rectilinear segment P1P2 205 into two regions (region 0 and region 1). When the pixels in region 0 have similar motion, the motion vector may describe the motion of all pixels in this region. The motion vector can be used to compress region 0. Similarly, when pixels in region 1 have similar motion, the associated motion vector can describe the motion of pixels in region 1. Such a geometric partition can be signaled to a receiver (e.g., a decoder) by encoding positions P1 and P2 (or representations of positions P1 and P2) in the video bitstream.

Still referring to Fig. 2, when encoding video data using geometric partitioning, a rectilinear segment 205 (or, more specifically, points P1 and P2) may be defined. However, a rectilinear segment may not be able to partition a block in a way that reflects the boundaries of an object. As a result, partitioning with rectilinear segments may not be able to provide an efficient partition of a block (e.g., in a way that any resulting remainder is small). This may be true in cases where a block may contain pixels (e.g., luminance samples) representing an object or a boundary having a curved (e.g., non-straight) boundary. For example, Fig. 3 shows an example embodiment of an image containing an apple that cannot be efficiently partitioned with rectilinear segments, wherein the image of the apple, as shown, includes several rectangular blocks indicating portions of the image, wherein if they are partitioned using a rectilinear segment according to the geometric partitioning, the partition may not accurately follow the boundary of the object (e.g., the apple), which, as shown in Fig. 3, is a curve.

Fig. 4A is a diagram illustrating a non-limiting example of exponential division using a non-linear curve, defined for the purposes of the present disclosure as a curve that is not a straight line, according to some aspects of the present subject matter, which allows for increased compression efficiency. A rectangular block 400 may include pixels (e.g., brightness samples). The rectangular block 400 may have, as a non-limiting example provided for illustrative purposes only, pixels of size 8×8 (e.g., brightness samples) or more.

In Fig. 4A, the rectangular block 400 can be divided into two or more regions, shown for illustrative purposes in Fig. 4A as region 0 and region 1, designated at 410 and 415, respectively, by the curved line 405. All of the brightness samples in each region defined in this way can be considered to have the same or similar motion and, thus, can be represented using the same motion vector. To illustrate exemplary purposes, it can be considered that all of the brightness samples in region 410 have the same or similar motion and can be represented by the same motion vector. Similarly, it can be considered that all of the brightness samples in region 415 have the same or similar motion and can be represented by the same motion vector. As will be described in more detail below, the respective motion vectors can be determined in accordance with the AMVP mode or the fusion mode. In some implementations and for purposes of discussion, all luminance samples to the left of or above the curved segment 405 dividing the rectangular block 400 may be considered to belong to region 0 (410). In some implementations, all luminance samples to the right of or below the curved segment 405 dividing the rectangular block 400 may be considered to belong to region 1 (415). In some implementations, all luminance samples through which the curved segment dividing the rectangular block 400 passes (i.e., luminance samples on and/or intersected by the rectilinear segment) belong to region 0 (410). In some implementations, all luminance samples through which the curved segment dividing the rectangular block 400 passes may be considered to belong to region 1 (415). Other implementations are possible, as will be apparent to those skilled in the art after reading the present disclosure as a whole.

Still referring to Fig. 4A, the exponential division may be represented in the bitstream. In some implementations, the exponential division mode may be used and the corresponding parameters may be transmitted in the bitstream. For example, the exponential division may be represented in the bitstream by signaling specified exponential division patterns. Fig. 4B shows a number of diagrams illustrating non-limiting examples of pattern sections 420-435. In some implementations, the signaling may be performed by including an index into one or more of these regular (e.g., patterned) exponential sections that have been specified. These regular exponential divisions may indicate a set of specified orientations. For example, Fig. 4C shows non-limiting examples of curves associated with 4 specified patterns (1, 2, 3, 4). In some implementations, the number of bends in the pattern may vary.

Still referring to Fig. 4B and as another non-limiting example, exponential sections may be represented in the bitstream by signaling specified coefficients, such as coefficients of exponential functions that indicate the degree of curvature that may allow additional exponential functions.

In some implementations, still referring to Fig. 4B, a given pattern from the plurality of patterns 420-435 used in the exponential partitioning mode may indicate a straight line segment. For example, in Fig. 4C, the segment indexed by a factor of 1 is a straight line, which can be considered a special case of exponential partitioning, that is, the geometric partitioning described above.

In some implementations, both orientation patterns, such as those shown in Fig. 4B, and predetermined patterns, such as those shown in Fig. 4C, may be used to efficiently signal any exponential pattern of a plurality of potential exponential partitions; for example, the patterns may provide curve options 440, including, but not limited to, a straight-line segment 1 and/or one or more non-linear curves 2-4, which may include exponential curves, each of which may be selected by an encoder, a user, and/or by an automated partition creation process, as described in this disclosure.

In some implementations, still referring to Fig. 4A, the indices of the starting point and the ending point may be predetermined. For example, Fig. 4A shows an example curved segment that starts at the lower left corner of the rectangular block 400 and ends at the upper right corner of the rectangular block 400. Such predetermined indices of the starting point and the ending point may be stored in the decoder memory. Alternatively or additionally, in some implementations, the indices of the starting point and the ending point may be explicitly signaled in the bitstream. For example, Fig. 4D shows another example block showing different indices of the starting point P1 and the ending point P2 that divide the rectangular block 400. The indices of the starting point P1 and the ending point P2 may be signaled directly or may be indicated by an index in a set of predetermined values. Other parameters are also possible, which will be understood by those skilled in the art after reading the present disclosure as a whole.

Still referring to Fig. 4A, inter-frame prediction can be performed using regions that have been exponentially divided. Motion vectors for motion compensation can be obtained using AMVP or a fusion mode. In the AMVP mode, motion vector prediction can be performed by signaling an index into a motion vector candidate list, and a motion vector difference (e.g., a residual) can be encoded and included in a bitstream. In the fusion mode, a motion vector is selected from a motion vector candidate list without encoding a motion vector difference, thereby allowing the current block to receive motion information of another previously decoded block. In both the AMVP and fusion modes, the candidate list can be generated by both the encoder and the decoder, and the index into the candidate list can be signaled in the bitstream.

Fig. 5 is a diagram illustrating non-limiting examples of locations of potential spatial motion vector candidates relative to an exemplary current block 1100 partitioned according to an exponential partition. The potential spatial motion vector candidates may be considered to form a list of motion vector candidates in the AMVP mode or the merging mode. As a non-limiting example, the current block 1100 may be partitioned into two regions, region S0 and region S1, by a curve between points P0 and P1. Each of region S0 and region S1 may be predicted in one or two directions. Spatial candidates for the first region (region S0) are shown for illustrative purposes in Fig. 5 and may include, without limitation, lower left candidate A0, left candidate A1, upper left candidate B2, upper candidate B1, and upper right candidate B0.

As shown in Fig. 5, and in some implementations, each location (A0, A1, B2, B1, and B0) may represent a block at the corresponding location. For example, and without limitation, the upper left candidate B2 may represent a block that is at a location that is immediately to the left of and immediately above the region S0; for example, if the brightness location of the upper left corner of S0 is (0, 0), then the upper left candidate may be at the location (-1, -1). The lower left candidate A0 may be located immediately to the left of P1; for example, and without limitation, if the brightness location of P1 is (P1x, P1y), the lower left candidate A0 may be at the location (P1x-1, P1y+1). The left candidate A1 may be located immediately to the left of P1; for example, the left candidate A1 may reside at the location (P1x-1, P1y). The upper candidate B1 may be located immediately above P0; or, for example, if the location of the brightness P0 is (P0x, P0y), the upper candidate B1 is located at (P0x, P0y-1). The upper right candidate B0 may be located immediately above the uppermost and rightmost brightness location in the second region S1; or, for example, if the upper right corner of S1 is located at (S0_width+S1_width-1, 0), then the upper right candidate B0 may be located at (S0_width+S1_width-1, -1), where M = S0_width+S1_width. Other locations are also possible, as will be understood by those skilled in the art after reading the present disclosure as a whole. Fig. 6 illustrates Fig. 5 with an annotation showing brightness locations, including the upper leftmost brightness location of the first region S0 and the upper rightmost brightness location of the second region S1.

In some implementations, still referring to Fig. 6, when forming the candidate list for the region S0, some of the potential candidates may be automatically marked as unavailable and removed from the list, since in exponential partitioning such partitioning may be performed to separate regions (or objects) in the frame that have different motion information. Accordingly, it may be concluded that the blocks associated with these candidates are likely to represent a different object with a different motion, and therefore these candidates may be automatically marked as unavailable (e.g., not further considered, removed from the list, etc.). As a non-limiting example and as shown above in Fig. 5, for the region S0, the lower left candidate A0 may be automatically marked as unavailable, since it is likely that the region S0 does not share motion information with the block located in the lower left candidate A0. Similarly, for region S0, the top-right candidate B0 can be automatically marked as unavailable, since it is likely that region S0 does not share motion information with the block located in the top-right candidate B0.

Fig. 7 is a diagram illustrating non-limiting examples of locations of potential motion vector candidates relative to an exemplary current block 1400 partitioned according to an exponential partition. The possible motion vector candidates may be considered to form a candidate list in an AMVP mode or a merging mode. The current block 1400 could be partitioned into two regions, region S0 and region S1, by a curve between points P0 and P1. Each of region S0 and region S1 may be predicted in one or two directions. Candidates for the second region (region S1) are shown in Fig. 7 and may include a lower left candidate A0, a left candidate A1, an upper left candidate B2, an upper candidate B1, and an upper right candidate B0.

As shown in Fig. 7, each location (A0, A1, B2, B1, and B0) may represent a block at the corresponding location. For example, the upper-left candidate B2 may be a block that is at a brightness location that is immediately to the left of and immediately above region S1; for example, if the brightness location at the upper-left corner of S1 is adjacent to P0 with brightness location coordinates of (P0x+1, P0y), then the upper-left candidate B2 may be at location (P0x, P0y-1). The lower-left candidate A0 may be located immediately to the left of and below the lower-leftmost brightness location of S1; for example, if the lower-leftmost brightness location of S1 is brightness location (0, S0_height+S1_height-1), the lower-left candidate A0 may be at location (-1, S0_height+S1_height), where N = S0_height+S1_height. The left candidate A1 may be located immediately to the left of the lower-leftmost luminance location of S1 (e.g., the lower-left corner of S1); for example, if the lower-leftmost luminance location of S1 is the luminance location (0, S0_height+S1_height-1), the left candidate A1 may be located at the luminance location (-1, S0_height+1_height-1). The top candidate B1 may be located immediately above the upper-rightmost luminance location of S1; for example, if the upper-rightmost luminance location of S1 is (S0_width+S1_width-1, 0), then the upper candidate B1 may be located at (S0_width+S1_width-1, -1), where M = S0_width+S1_width. The upper-right candidate B0 may be located immediately above and to the right of the upper-rightmost luminance location of the second region S1; for example, if the rightmost upper location of the brightness of S1 (e.g., the upper right corner) is located at (S0_width+S1_width-1, 0), then the rightmost upper candidate B0 may be located at (S0_width+S1_width, -1). Fig. 8 illustrates Fig. 7 with an annotation showing brightness locations, including the leftmost lower location of the brightness of the second region of S1 and the rightmost upper location of the brightness of the second region of S1.

In some implementations, still referring to Fig. 8, when forming the candidate list for the region S1, some potential candidates may be automatically marked as unavailable and removed from the list, since in exponential partitioning such partitioning may be performed to divide into regions (or objects) in the frame that have different motion information. Accordingly, it may be concluded that the blocks associated with these candidates are likely to represent a different object with a different motion, and therefore these candidates may be automatically marked as unavailable (e.g., as not further considered, removed from the list, etc.). In the example shown in Fig. 7, for the region S1, the upper left candidate B2 may be automatically marked as unavailable, since it is likely that the region S1 does not share motion information with the block located in the upper left candidate B2.

Fig. 9 is a block diagram of a system illustrating a non-limiting example of a video encoder 900 configured to encode video using inter-picture prediction with exponential partitioning. An example video encoder 900 receives an input video 905 that may initially be segmented or partitioned according to a processing scheme, such as a tree-structured coding block partitioning scheme (e.g., a quad-tree plus binary tree (QTBT)). An example tree-structured coding block partitioning scheme may include partitioning a picture frame into large block elements called coding tree units (CTUs). In some implementations, each CTU may be further partitioned one or more times into a number of sub-blocks called coding units (CUs). The end result of this partitioning may include a group of sub-blocks, which may be called prediction units (PUs). In addition, transform units (TUs) may also be used. Such a partitioning scheme may include performing exponential partitioning according to some aspects of the present subject matter. Fig. 8 shows an example of QTBT frame partitioning, and Fig. 11 shows an example of exponential partitioning at the CU level of QTBT shown in Fig. 8.

Still referring to Fig. 9, an example video encoder 900 may include an intra-frame prediction processor 915, a motion estimation/compensation processor 920 (also referred to as an inter-frame prediction processor) configured to support exponential partitioning, including an AMVP mode and a fusion mode, a transform/quantization processor 925, an inverse quantization/inverse transform processor 930, an in-loop filter 935, a decoded picture buffer 940, and an entropy coding processor 945. In some implementations, the motion estimation/compensation processor 920 may perform inter-frame prediction using exponential partitioning, including using the AMVP mode and the fusion mode. Bitstream parameters that signal the exponential partitioning modes, the AMVP mode, and the fusion mode may be input to the entropy coding processor 945 for inclusion in the output bitstream 950.

In the process of operation, still referring to Fig. 9, for each block of the frame of the input video 905, it can be determined whether the block is to be processed using intra-frame image prediction or using motion estimation/compensation. The block can be provided to the intra-frame prediction processor 910 or the motion estimation/compensation processor 920. If the block is to be processed using intra-frame prediction, the intra-frame prediction processor 910 can perform processing to output a predictor. If the block is to be processed using motion estimation/compensation, the motion estimation/compensation processor 920 can perform processing including using exponential division with the AMVP mode and the fusion mode to output the predictor.

Still referring to Fig. 9, a residual can be formed by subtracting a predictor from the input video. The residual can be received by a transform/quantization processor 925, which can perform transform processing (e.g., a discrete cosine transform (DCT)) to obtain coefficients that can be quantized. The quantized coefficients and any associated signaling information can be provided to an entropy encoding processor 945 for entropy encoding and inclusion in the output bitstream 950. The entropy encoding processor 945 can support encoding of signaling information related to the exponential division modes, the AMVP mode, and the merge mode. In addition, the quantized coefficients can be provided to an inverse quantization/inverse transform processor 930, which can produce pixels that can be combined with a predictor and processed by a loop filter 935, the output of which can be stored in a decoded picture buffer 940 for use by a motion estimation/compensation processor 920, which can support exponential division modes, an AMVP mode, and a fusion mode.

Fig. 12 is a flow chart illustrating an example process 1200 for encoding video with exponential partitioning and inter-frame prediction according to some aspects of the present subject matter, which allows for reducing encoding complexity while increasing compression efficiency. At step 1210, a video frame may undergo initial block segmentation, for example, using a tree-structured coding block partitioning scheme, which may include partitioning a picture frame into CTUs and CUs. At step 1220, a block may be selected for exponential partitioning, wherein the exponential partitioning may include geometric partitioning or may include exponential partitioning using a nonlinear curve. The selection may include identifying, according to a metric rule, that the block should be processed in accordance with the exponential partitioning mode.

At step 1230, still with reference to Fig. 12, an exponential division may be defined. A curved and/or linear segment (e.g., 405) and/or a rectilinear and/or linear segment that will divide the pixels contained in the block according to their movement between frames into two non-rectangular regions (e.g., region 0 and region 1) may be defined such that the pixels (e.g., brightness samples) in one of the regions (e.g., in region 0) have a similar movement, and the pixels in the other region (e.g., in region 1) have a similar movement.

At step 1240, still referring to FIG. 12 , the motion information of each non-rectangular region can be determined and processed using the AMVP mode or the merge mode. When processing the region using the AMVP mode, a candidate list can be formed by considering both spatial and temporal candidates, including spatial candidates as described above, and including marking some candidates as unavailable. A motion vector can be selected from the motion vector candidate list as a motion vector prediction, and a motion vector difference (e.g., a residual) can be calculated. An index can be defined in the candidate list. In the merge mode, the candidate list can be formed by considering both spatial and temporal candidates, including the spatial candidates described above, and including marking some candidates as unavailable. A motion vector can be selected from the motion vector candidate list for the region to obtain motion information of another block. An index can be defined in the candidate list.

At step 1250, still referring to Fig. 12, information about the determined exponential section and the motion can be transmitted in the bitstream. Signaling exponential sections in the bitstream can include, for example, including an index in one or more specified patterns and/or coefficients. Signaling motion information when processing an area using AMVP can include including a motion vector difference (e.g., a residual) and an index in a list of motion vector candidates in the bitstream. Signaling motion information when processing an area using the merge mode can include including an index in a list of motion vector candidates in the bitstream.

Fig. 13 is a block diagram of a system illustrating an example decoder 600 configured to decode a bitstream 1370 using exponential partitioning and inter-frame prediction according to some aspects of the present subject matter. The decoder 600 may include an entropy decoder processor 1310, an inverse quantization and inverse transform processor 1320, a deblocking filter 1330, a frame buffer 1340, a motion compensation processor 1350, and an intra-frame prediction processor 1360. In some implementations, the bitstream 1370 includes parameters that signal an exponential partitioning mode, an AMVP mode, and a merging mode. The motion compensation processor 1350 may recover pixel information using exponential partitioning and inter-frame prediction, as described in this disclosure.

During operation, the bit stream 1370 can be received by the decoder 600 and can be supplied to the entropy decoder processor 1310, which entropy decodes the bit stream into quantized coefficients. The quantized coefficients can be provided to the inverse quantization and inverse transform processor 1320, which can perform inverse quantization and inverse transform to create a residual signal, which can be added to the output signal of the motion compensation processor 1350 or the intra-frame prediction processor 1360 in accordance with the processing mode. The output signal of the motion compensation processor 1350 and the intra-frame prediction processor 1360 can include a prediction of a block based on a previously decoded block. The sum of the prediction and the residual can be processed by the deblocking filter 1330 and stored in the frame buffer 1340. For a given block (e.g., CU or PU), when bitstream 1370 signals that the partitioning mode is exponential partitioning, motion compensation processor 1350 may construct a prediction based on the exponential partitioning approach described herein and using AMVP or merging modes as described herein.

Fig. 14 is a flow chart illustrating an example process 1400 for decoding a bitstream using exponential partitioning and inter-frame prediction according to some aspects of the present subject matter. At step 1410, a bitstream is received. The reception may include extracting and/or parsing the bitstream and associated signaling information from the bitstream, including parsing a current block and associated signaling information from the bitstream.

In step 1420, still referring to Fig. 14, the current block may be divided using an exponential division mode into a first region and a second region. The division may include determining whether the exponential division mode is enabled (e.g., true) for the block, which indicates that exponential division is applied using nonlinear curves. If the exponential division mode is not enabled (e.g., the value "false" is selected), the decoder may process the block using an alternative exponential division mode, such as geometric division, where parameters for geometric division, including, but not limited to, end points, line segments, coefficients or the like, may be received from the bitstream, as described above. If the exponential division mode is enabled (e.g., the value "true" is selected), the decoder may extract or determine one or more parameters that characterize the exponential division. These parameters may include, for example, indices of exponential coefficients, values of exponential coefficients, indices of an orientation pattern, and/or indices of the beginning and end of a curved line (e.g., P1P2). Extraction or determination may include identification and extraction of parameters from a bitstream (e.g., parsing the bitstream).

At step 1430, still referring to Fig. 14, a motion vector associated with a region from the first region or the second region may be determined. Determining the motion vector may include determining whether the motion information of the region should be determined using the AMVP mode or the merge mode. When processing the region using the AMVP mode, a candidate list may be generated taking into account both spatial and temporal candidates, including the spatial candidates described above, and including taking into account marking some candidates as unavailable. A motion vector may be selected from the motion vector candidate list as a motion vector prediction, and a motion vector difference (e.g., a residual) may be calculated. In the merge mode, the determining may include generating a candidate list of spatial candidates and temporal candidates for each region. For each region, the spatial candidates may be spatial candidates as described above with reference to Figs. 5-8. Generating the candidate list may include automatically marking candidates as unavailable and removing unavailable candidates from the candidate list. The index into the generated candidate list can be parsed from the bitstream and can be used to select the final candidate from the candidate list. The motion information for the current region can be defined in the same way as the motion information of the final candidate (e.g., the motion vector for the region can be borrowed from the final candidate).

At step 1440, still referring to Fig. 14, the current block may be decoded using the determined motion vector.

Still referring to Fig. 14, although some variants have been described in detail above, other modifications or additions are possible. For example, in some implementations, exponential partitioning may be applied to symmetric blocks (8x8, 16x16, 32x32, 64x64, 128x128, etc.) as well as to various asymmetric blocks (8x4, 16x8, etc.).

Still referring to Fig. 14, the partitioning can be signaled in the bitstream based on rate-distortion decisions in the encoder. The coding can be based on a combination of regular predefined partitions (e.g., patterns), spatiotemporal partition prediction, and additional offsets. Each exponentially partitioned region can use motion-compensated prediction or intra-frame prediction. The boundary of the predicted regions can be smoothed before adding the residual. For the residual coding, the encoder can choose between a conventional rectangular DCT for the entire block and a shape-adaptive DCT for each region.

Still referring to Fig. 14, in some implementations, a quadtree plus binary decision tree (QTBT) may be implemented. In QTBT, at the coding tree unit level, the QTBT splitting parameters can be obtained dynamically to adapt to local characteristics without transmitting any overhead. Then, at the coding unit (CU) level, the decision tree structure of the joint classifier can eliminate unnecessary iterations and control the risk of false prediction. In some implementations, exponential splitting may be available as an additional splitting option available at each leaf node of QTBT. In some implementations, exponential splitting is available as an additional coding tool at the CU level of QTBT splitting. For example, Fig. 8 shows an example of QTBT splitting of a frame, and Fig. 11 shows an example of exponential splitting at the CU level of QTBT shown in Fig. 8.

In some implementations, the decoder includes an exponential partition processor that can generate an exponential partition for the current block and provide all partition-related information to dependent processes. The exponential partition processor can directly affect motion compensation, since it can be performed segment by segment in case the block is partitioned exponentially. In addition, the partition processor can provide shape information to the intra-frame prediction processor and the transform coding processor.

In some implementations, additional syntax elements may be signaled at different hierarchical levels of the bitstream. In order to enable exponential partitioning for the entire sequence, an enable flag may be encoded in the sequence parameter set (SPS). In addition, a CTU flag may be encoded at the coding tree unit (CTU) level to indicate whether any coding units (CUs) use exponential partitioning. The CU flag may be encoded to indicate whether the current coding unit uses exponential partitioning. Parameters that specify a curved line on a block may be encoded. For each region, a flag may be decoded that specifies whether the current region is predicted using inter-frame or intra-frame prediction.

In some implementations, the minimum region size may be precisely defined.

The subject matter described in this document provides numerous technical advantages. For example, some implementations of the current subject matter may provide for block partitioning, which increases compression efficiency. In some implementations, implementing partitioning in a manner that more closely follows object boundaries can achieve effective visual effects. Similarly, in some implementations, implementing partitioning in a manner that more closely follows object boundaries can reduce blocking artifacts at object boundaries.

It should be noted that any one or more aspects and embodiments described herein can be readily implemented using digital electronic circuits, integrated circuits, application-specific integrated circuits (ASICs) designed specifically for specific applications, computer hardware with field-programmable gate arrays (FPGAs), firmware, software, and/or combinations thereof, as implemented and/or implemented in one or more machines (e.g., one or more computing devices that are used as a user computing device for an electronic document, one or more server devices, such as a document server, etc.), programmed in accordance with the inventive ideas presented herein, as will be apparent to those skilled in the art of computer technology. These various aspects or features may include an implementation in one or more computer programs and/or software that are executable and/or inter-frame programmable in a programmable system that includes at least one programmable processor, which may be special-purpose or general-purpose and may be connected to receive data and instructions from a storage system and to transmit data and instructions to the storage system, to at least one input device and to at least one output device. The corresponding software coding can be easily prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software field. The aspects and implementations discussed above using software and/or software modules may also include corresponding hardware to facilitate the implementation of computer-executable instructions of the software and/or software module.

Such software may be a computer program product that uses a machine-readable storage medium. A machine-readable storage medium may be any storage medium that is capable of storing and/or encoding a sequence of instructions for execution by a machine (e.g., a computing device) and that cause the machine to perform any of the methodologies and/or embodiments described in this document. Examples of a machine-readable storage medium include, but are not limited to, a magnetic disk, an optical disk (e.g., CD, CD-R, DVD, DVD-R, etc.), a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), a magnetic card, an optical card, a solid-state memory, EPROM, EEPROM, programmable logic devices (PLD), and/or any combinations thereof. As used herein, the term "machine-readable storage medium" is intended to include a single storage medium as well as a collection of physically separate storage media, such as, for example, a collection of compact discs or one or more hard disks in combination with computer memory. As used herein, the term "machine-readable storage medium" does not include transient forms of signal transmission.

Such software may also include information (e.g., data) carried as a data signal on a carrier, such as a carrier. For example, computer-executable information may be included as a data-carrying signal embodied in the carrier, where the signal encodes a sequence of instructions or a portion thereof for execution by a machine (e.g., a computing device) and any associated information (e.g., data structures and data) that cause the machine to perform any of the methodologies and/or embodiments described herein.

Examples of a computing device include, but are not limited to, an e-book reader, a computer workstation, a terminal computer, a server computer, a portable device (e.g., a tablet computer, a smartphone, etc.), a web device, a network router, a network switch, a network bridge, any computer capable of executing a sequence of instructions specifying an action to be performed on that computer, and any combination thereof. In one example, a computing device may include and/or be included in a kiosk.

Fig. 15 shows a schematic representation of one embodiment of a computing device in the form of an example computer system 1500 that can execute a set of instructions for causing a control system to perform any one or more aspects and/or methodologies of the present disclosure. Moreover, it is also contemplated that multiple computing devices can be used to implement a specially configured set of instructions that cause one or more devices to perform any one or more aspects and/or methodologies of the present disclosure. Computer system 1500 includes a processor 1504 and a memory 1508 that communicate with each other and with other components via a bus 1512. Bus 1512 can include any of several types of bus structures, including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combinations thereof that use any of a variety of bus architectures.

Memory 1508 may include various components (e.g., computer-readable storage media), including, but not limited to, a random access memory component, a read-only component, and any combinations thereof. In one example, a basic input/output system (BIOS) 1516, including basic routines that help transfer information between elements in computer system 1500, such as during startup, may be stored in memory 1508. Memory 1508 may also include (e.g., stored on one or more computer-readable storage media) instructions (e.g., software) 1520 that embody any one or more aspects and/or methodologies of the present disclosure. In another example, memory 1508 may further include any number of program modules, including, but not limited to, an operating system, one or more application programs, other program modules, program data, and any combinations thereof.

Computer system 1500 may also include storage device 1524. Examples of storage device (e.g., storage device 1524) include, but are not limited to, a hard disk drive, a magnetic disk drive, an optical disk drive in combination with an optical storage medium, a solid-state memory device, and any combinations thereof. Storage device 1524 may be connected to bus 1512 via a corresponding interframe face (not shown). Examples of interframe faces include, but are not limited to, SCSI, Advanced Attachment Technology (ATA), serial ATA, Universal Serial Bus (USB), IEEE 1394 (FIREWIRE), and any combinations thereof. In one example, storage device 1524 (or one or more components thereof) may be connected via an interframe face to computer system 1500 (e.g., via an external port connector (not shown)). In particular, the memory device 1524 and the associated computer-readable storage medium 1528 may provide non-volatile and/or non-volatile storage of computer-readable instructions, data structures, program modules and/or other data for the computer system 1500. In one example, the software 1520 may reside, in whole or in part, on the computer-readable storage medium 1528. In another example, the software 1520 may reside, in whole or in part, in the processor 1504.

The computer system 1500 may also include an input device 1532. In one example, a user of the computer system 1500 may enter commands and/or other information into the computer system 1500 via the input device 1532. Examples of the input device 1532 include, but are not limited to, an alphanumeric input device (e.g., a keyboard), a pointing device, a joystick, a game controller, an audio input device (e.g., a microphone, a voice response system, etc.), a cursor control device (e.g., a mouse), a touch pad, an optical scanner, a video capture device (e.g., a camera, a video camera), a touch screen, and any combinations thereof. The input device 1532 may be connected via an interframe face to the bus 1512 via any of a plurality of interframe faces (not shown), including, but not limited to, a serial interframe face, a parallel interframe face, a game port, a USB interframe face, a FIREWIRE interframe face, a direct interframe face to the bus 1512, and any combinations thereof. The input device 1532 may include a touch screen interframe face, which may be part of the display 1536 or be separate from it, which is discussed further below. The input device 1532 may be used as a user selection device for selecting one or more graphical representations in a graphical interframe face, as described above.

A user may also enter commands and/or other information into the computer system 1500 via a storage device 1524 (e.g., a removable disk, a flash disk, etc.) and/or a network interframe face device 1540. A network interframe face device, such as a network interframe face device 1540, may be used to connect the computer system 1500 to one or more of a plurality of networks, such as a network 1544, and to one or more remote devices 1548 connected thereto. Examples of a network interframe face device include, but are not limited to, a network interframe face card (e.g., a mobile network interframe face card, a LAN card), a modem, and any combinations thereof. Examples of a network include, but are not limited to, a network (e.g., an interframe network, an organizational network), a local area network (e.g., a network associated with an office, building, campus, or other relatively small geographic area), a telephone network, a data network associated with a telephone/voice provider (e.g., a mobile phone provider's data and/or a telephone network), a direct connection between two computing devices, and any combinations thereof. A network, such as network 1544, may use a wired and/or wireless communication mode. In general, any network topology may be used. Information (e.g., data, software 1520, etc.) may be transmitted to and/or from computer system 1500 using the network interframe face of device 1540.

The computer system 1500 may further include a video display adapter 1552 for transmitting a displayed image to a display device, such as a display device 1536. Examples of a display device include, but are not limited to, a liquid crystal display (LCD), a cathode ray tube (CRT), a plasma display, a light emitting diode (LED), and any combinations thereof. The display adapter 1552 and the display device 1536 may be used in combination with the processor 1504 to provide a graphical representation of aspects of the present disclosure. In addition to the display device, the computer system 1500 may include one or more other peripheral output devices, including, but not limited to, an audio speaker, an interframe, and any combinations thereof. Such peripheral output devices may be connected to bus 1512 via peripheral interframe face 1556. Examples of peripheral interframe faces include, but are not limited to, a serial port, a USB connection, a FIREWIRE connection, a parallel connection, and any combinations thereof.

The foregoing has described in detail illustrative embodiments of the present invention. Various modifications and additions can be made without departing from the spirit and scope of the present invention. The features of each of the various embodiments described above can be combined with features of other described embodiments, where appropriate, to provide a variety of combinations of features in associated new embodiments. Furthermore, while a number of specific embodiments have been described above, what has been described herein is merely illustrative of the application of the principles of the present invention. In addition, while specific methods herein may be illustrated and/or described as being performed in a particular order, the order of performance varies widely within the ordinary skill in achieving the embodiments disclosed herein. Accordingly, this description is intended to be used as an example only and does not otherwise limit the scope of the present invention.

In the above descriptions and claims, phrases such as "at least one of" or "one or more" may appear followed by a conjunctive list of elements or features. The term "and/or" may also appear in a list of two or more elements or features. Unless otherwise implicitly or clearly contradicted by the context in which it is used, such a phrase means any of the listed elements or features alone, or any of the listed elements or features in combination with any of the other listed elements or features. For example, the phrases "at least one of A and B," "one or more of A and B," and "A and/or B" each mean "A taken alone," "B taken alone," or "A and B together." Similar interframe representations are also intended for lists of three or more items. For example, each of the phrases "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, and/or C" means "A taken alone," "B taken alone," "C taken alone," "A and B together," "A and C together," "B and C together," or "A and B and C together." In addition, the use of the term "based on" in the description above and in the claims means "based at least in part," so an unspecified feature or element is also permissible.

The subject matter described herein may be embodied in systems, devices, methods and/or articles depending on the desired configuration. The embodiments set forth in the preceding description do not represent all embodiments consistent with the subject matter described herein. Instead, they represent only some examples consistent with aspects related to the described subject matter. Although some embodiments have been described in detail above, other modifications or additions are possible. In particular, additional features and/or embodiments may be provided in addition to those set forth in this description. For example, the embodiments described above may be directed to various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of several additional features disclosed above. In addition, the logical flows depicted in the accompanying drawings and/or described herein do not necessarily require the particular order shown or the sequential order to achieve the desired results. Other embodiments may fall within the scope of the following claims.

Claims (24)

1. A decoder capable of: receiving a bit stream including an encoded image, wherein the encoded image includes a block of encoded pixels and signaling information including a first division index used to determine a starting point of an indirect non-rectangular division and a second division index used to determine an ending point of an indirect non-rectangular division; determining, using a first partition index, a starting point of an indirect non-rectangular partition in the block, wherein the starting point is located on a first side of the block at a first offset distance from a first corner of the block; determining, using a second partition index, an end point of the indirect non-rectangular partition in the block, wherein the end point is located on a second side of the block at a second offset distance from a second corner of the block; generating first prediction values of pixels in a first region on a first side of a relatively indirect non-rectangular partition; generating second prediction values of pixels in a second region on a second side of the relatively indirect non-rectangular partition; and decoding a block using pixel prediction values. 2. The decoder according to claim 1, wherein a first motion vector selected from a first set of motion vector candidates is used to generate the first region of pixel prediction values, and a second motion vector selected from a second set of motion vector candidates is used to generate the second region of pixel prediction values. 3. The decoder according to item 1, characterized in that it is additionally configured with the ability to: smoothing the first pixel prediction values and the second pixel prediction values along an indirect non-rectangular partition; and decoding the encoded block by adding the residual pixel values to the smoothed first and second pixel prediction values. 4. The decoder according to claim 1, wherein the block is a unit of a coding tree. 5. The decoder of claim 1, wherein the block has a size of N×N, where N is equal to one of 128, 64, and 32. 6. The decoder of claim 1, wherein the indirect non-rectangular partition is a curve. 7. A decoder configured to: receiving a bit stream including an encoded image, wherein the encoded image includes a coding tree unit containing a plurality of coding units and signaling information including a first partition index used to define a starting point of an indirect non-rectangular partition in the coding tree unit and a second partition index used to define an ending point of an indirect non-rectangular partition in the coding tree unit; determining, using a first partition index and a second partition index, a starting point and an ending point of an indirect non-rectangular partition in a coding tree unit; generating first prediction values of pixels in a first region on a first side of a relatively indirect non-rectangular partition, using a first motion vector from a first set of candidate motion vectors; generating second prediction values of pixels in a second region on a second side of the relatively indirect non-rectangular partition, using a second motion vector from a second set of candidate motion vectors; smoothing the first pixel prediction values and the second pixel prediction values along an indirect non-rectangular partition; and decoding a unit of the coding tree by adding residual pixel values to the smoothed first and second pixel prediction values. 8. The decoder according to item 7, in which: the bitstream includes a first motion vector candidate index for identifying a first motion vector in a first set of motion vector candidates; and the bitstream includes a second motion vector candidate index for identifying a second motion vector in a second set of motion vector candidates.