WO2021253373A1 - Partitionnement géométrique probabiliste en codage de vidéo - Google Patents

Partitionnement géométrique probabiliste en codage de vidéo Download PDF

Info

Publication number
WO2021253373A1
WO2021253373A1 PCT/CN2020/096988 CN2020096988W WO2021253373A1 WO 2021253373 A1 WO2021253373 A1 WO 2021253373A1 CN 2020096988 W CN2020096988 W CN 2020096988W WO 2021253373 A1 WO2021253373 A1 WO 2021253373A1
Authority
WO
WIPO (PCT)
Prior art keywords
template
partitioning
regard
current block
probability
Prior art date
Application number
PCT/CN2020/096988
Other languages
English (en)
Inventor
Tsuishan CHANG
Yuchen SUN
Jian Lou
Original Assignee
Alibaba Group Holding Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Limited filed Critical Alibaba Group Holding Limited
Priority to PCT/CN2020/096988 priority Critical patent/WO2021253373A1/fr
Publication of WO2021253373A1 publication Critical patent/WO2021253373A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock

Definitions

  • VVC Versatile Video Coding
  • GEO geometric partitioning
  • AVC Advanced Video Coding
  • H. 265/HEVC High Efficiency Video Coding
  • GEO to efficiently code boundaries and edges of objects in a picture, any particular block of a picture may be internally partitioned into two irregular partitions by a partitioning line spanning two edges of the block.
  • GEO provides for predefined sets of unique internal partitionings of blocks and sub-blocks of various dimensions, enabling boundaries and edges in a picture to be described accurately in a granular manner.
  • This proposal provides an improvement over column-and row-based partitioning, enabling edges, especially moving edges, to be coded within blocks instead of between blocks and therefore enhancing sharpness and decreasing “staircase” effects in displayed pictures around edges.
  • implementations of GEO require potentially determining an internal partition within each block of a picture, incurring compounding computing overhead costs with the number of blocks coded in the picture. Consequently, it is desirable to alleviate such computational overheads to avoid greatly exacerbating encoding time of video content coded according to GEO proposals.
  • FIG. 1A illustrates multiple preset angles which may serve as a basis for classifying partitionings according to implementations of geometric partitioning.
  • FIG. 1B illustrates multiple classifications of partitionings, each classification being a unique combination of an angle and a distance.
  • FIG. 1C illustrates multiple examples of partitionings as applied to coded blocks.
  • FIG. 2 illustrates an example block diagram of a video coding process according to an example embodiment of the present disclosure.
  • FIG. 3 illustrates an example of a blending formula applied to overlapping pixels of two weighting masks at a partitioning line.
  • FIG. 4 illustrates a flowchart of a geometric partitioning method implementing probabilistic thresholds according to example embodiments of the present disclosure.
  • FIG. 5A illustrates a flowchart of probabilistic search steps according to an example embodiment of the present disclosure.
  • FIGS. 5B, 5C, and 5D illustrate flowcharts of probabilistic search steps according to additional example embodiments of the present disclosure.
  • FIG. 6 illustrates an example system for implementing the processes and methods described above for implementing probabilistic geometric partitioning.
  • Systems and methods discussed herein are directed to implement geometric partitioning in a video encoder, and more specifically to mechanisms for probabilistically applying partitioning to individual blocks of frames based on implementations of a geometric partitioning during operation of a video encoder according to video coding standards.
  • a frame may be subdivided into macroblocks ( “MBs” ) each having dimensions of 16x16 pixels, which may be further subdivided into partitions.
  • MBs macroblocks
  • a frame may be subdivided into coding tree units ( “CTUs” ) , the luma and chroma components of which may be further subdivided into coding tree blocks ( “CTBs” ) which are further subdivided into coding units ( “CUs” ) .
  • CTUs coding tree units
  • CTBs coding tree blocks
  • a frame may be subdivided into units of NxN pixels, which may then be further subdivided into subunits.
  • Each of these largest subdivided units of a frame may generally be referred to as a “block” for the purpose of this disclosure.
  • a block may be subdivided into partitions having dimensions in multiples of 4x4 pixels.
  • a partition of a block may have dimensions of 8x4 pixels, 4x8 pixels, 8x8 pixels, 16x8 pixels, or 8x16 pixels.
  • motion prediction coding formats may refer to data formats wherein frames are encoded with motion vector information and prediction information of a frame by the inclusion of one or more references to motion information and prediction units ( “PUs” ) of one or more other frames.
  • Motion information may refer to data describing motion of a block structure of a frame or a unit or subunit thereof, such as motion vectors and references to blocks of a current frame or of another frame.
  • PUs may refer to a unit or multiple subunits corresponding to a block structure among multiple block structures of a frame, such as an MB or a CTU, wherein blocks are partitioned based on the frame data and are coded according to established video codecs.
  • Motion information corresponding to a PU may describe motion prediction as encoded by any motion vector coding tool, including, but not limited to, those described herein.
  • frames may be encoded with transform information by the inclusion of one or more transformation units ( “TUs” ) .
  • Transform information may refer to coefficients representing one of several spatial transformations, such as a diagonal flip, a vertical flip, or a rotation, which may be applied to a sub-block.
  • Sub-blocks of CUs such as PUs and TUs may be arranged in any combination of sub-block dimensions as described above.
  • a CU may be subdivided into a residual quadtree ( “RQT” ) , a hierarchical structure of TUs.
  • the RQT provides an order for motion prediction and residual coding over sub-blocks of each level and recursively down each level of the RQT.
  • blocks and sub-blocks of a frame may further be partitioned according to geometric partitioning ( “GEO” ) .
  • GEO geometric partitioning
  • Square or non-square blocks and sub-blocks of size having dimensions of at least 8 luma samples to each side may be partitioned according to GEO.
  • a partitioning of a block or sub-block according to GEO may be indicated by a straight partitioning line spanning a first coordinate of a first side of the block or sub-block and a second coordinate of a second side of the block.
  • an angle of the partitioning line as drawn from the first coordinate to the second coordinate, and a distance of the partitioning line as spanning the first coordinate and the second coordinate may be characterized.
  • the angle of the partitioning line and the distance of the partitioning line may further classify the partitioning as one of multiple template partitionings which may be specified according to implementations of GEO.
  • a partitioning may be classified based on some number of preset angles.
  • FIG. 1A illustrates multiple preset angles which may serve as a basis for classifying partitionings according to implementations of GEO.
  • the angles should be sufficiently apart to be discernable from each other within the minimum block size of 8x8 luma samples.
  • twenty of the angles as illustrated may be preset for classifying partitionings.
  • a partitioning may be classified based on some number of preset distances.
  • the distances should be sufficiently different in length to be discernable from each other within the minimum block size of 8x8 luma samples.
  • the number of preset distances cannot be arbitrarily many. For example, according to an implementation of GEO, four distances may be preset for classifying partitionings.
  • partitionings Based on twenty preset angles and four preset distances, up to eighty partitionings may potentially be classified. According to an implementation of GEO, however, a number of potential partitioning classifications are excluded due to non-uniqueness therebetween. Sixty-four unique combinations of angles and distances may be defined according to an implementation of GEO as bases for classifying partitionings. These classifications may be independent of the values of the first coordinate and the second coordinate which describe the partitioning line.
  • FIG. 1B illustrates multiple classifications of partitionings, each classification being a unique combination of an angle and a distance. Angles and distances may vary depending on whether a partitioning line spans opposite sides of a block, a corner of a block and a non-adjacent side of a block, or two adjacent sides of a block.
  • FIG. 1C illustrates multiple examples of partitionings as applied to coded blocks. Not all partitionings illustrated here exhibit unique combinations of an angle and a distance.
  • a video encoder implementing GEO may obtain a picture from a video source and code the frame to obtain a reconstructed frame that may be output for transmission.
  • Blocks of a reconstructed frame may be intra-coded or inter-coded.
  • Blocks such as those partitioned by GEO may be coded by skip mode or merge mode, wherein motion information of the region is copied from motion information of blocks of another reference frame.
  • the copied motion information may further be resized and re- sampled in proportion to a difference between a resolution of the current reference frame and a resolution of the other reference frame. Consequently, edges of objects which appear across multiple frames may be encoded using consistent image data, maintaining continuity and fluidity of motion across frames.
  • FIG. 2 illustrates an example block diagram of a video coding process 200 according to an example embodiment of the present disclosure.
  • a picture from a video source 202 may be encoded to generate a reconstructed frame, and the reconstructed frame may be output at a destination such as a reference frame buffer 204 or a transmission buffer 214.
  • the picture may be input into a coding loop, which may include the steps of inputting the picture into a first in-loop up-sampler or down-sampler 206, generating an up-sampled or down-sampled picture, inputting the up-sampled or down-sampled picture into a video encoder 208, generating a reconstructed frame based on a previous reconstructed frame of the reference frame buffer 204, inputting the reconstructed frame into one or more in-loop filters 210, and outputting the reconstructed frame from the loop, which may include inputting the reconstructed frame into a second up-sampler or down-sampler 212 generating an up-sampled or down-sampled reconstructed frame, and outputting the up-
  • a coded frame is obtained from a source such as a bitstream 218.
  • a source such as a bitstream 218.
  • a previous frame having position N–1 in the bitstream 218 may have a resolution larger than or smaller than a resolution of current frame
  • a next frame having position N+1 in the bitstream 218 may have a resolution larger than or smaller than the resolution of the current frame.
  • the current frame may be input into a coding loop, which may include the steps of inputting the current frame into a video decoder 220, inputting the current frame into one or more in-loop filters 222, inputting the current frame into a third in-loop up-sampler or down-sampler 224, generating an up-sampled or down-sampled reconstructed frame, and outputting the up-sampled or down-sampled reconstructed frame into the reference frame buffer 204.
  • the current frame may be output from the loop, which may include outputting the up-sampled or down-sampled reconstructed frame into a display buffer.
  • the video encoder 208 and the video decoder 220 may each implement a motion prediction coding format, including, but not limited to, those coding formats described herein.
  • Generating a reconstructed frame based on a previous reconstructed frame of the reference frame buffer 204 may include inter-coded motion prediction as described herein, wherein the previous reconstructed frame may be an up-sampled or down-sampled reconstructed frame output by the in-loop up-sampler or down-sampler 212/224, and the previous reconstructed frame serves as a reference picture in inter-coded motion prediction as described herein.
  • the video encoder 208 in generating a reconstructed frame, may record motion prediction information in data of the reconstructed frame.
  • Motion prediction information may include, for individual blocks and/or sub-blocks thereof, a partitioning based on one of the template partitionings provided by an implementation of GEO.
  • the partitioning may be defined by a partitioning line spanning a first coordinate and a second coordinate as described above.
  • a video encoder 208 may record motion prediction information over a frame on a per-block basis in a coding order among blocks of the frame, such as a raster scan order wherein a first-decoded block is an uppermost and leftmost block of the frame, according to video encoding standards. For each block, the video encoder 208 may look up, by an index or table, each template partitioning according to an implementation of GEO, and evaluate whether each template partitioning applies to the block according to image data of the block (i.e., whether pixels of the block may be divided by a line according to each template partitioning) .
  • the video encoder 208 may evaluate whether each template partitioning applies to the block according to partitionings applying to spatially or temporally neighboring blocks.
  • Multiple spatially neighboring blocks of a block of a frame may be present; spatially neighboring blocks may be blocks neighboring a left side of the current block and blocks neighboring a top side of the current block, since, by raster scan order, no other neighboring blocks will have been scanned and partitioned.
  • the video encoder 208 may evaluate a partitioning of the block.
  • the video encoder 208 may apply two weighting masks to the luma samples on either side of the partitioning line.
  • the partitioning line may indicate that luma samples on one side of the partitioning line differ in value from luma samples on the other side of the partitioning line; thus, the two weighting masks may include an up-weighting mask and a down-weighting mask, the up-weighting mask being applied to a side of the partitioning line having comparatively higher-valued luma samples to increase those values, and the down-weighting mask being applied to a side of the partitioning line having comparatively lower-valued luma samples to decrease those values to near zero.
  • the two weighting masks may overlap at the partitioning line.
  • a blending formula may be applied to the overlapping pixels, to derive a blended mask which weighs luma samples to values between those resulting from the up-weighting mask and those resulting from the down-weighting mask.
  • FIG. 3 illustrates an example of a blending formula applied to overlapping pixels of two weighting masks at a partitioning line.
  • the video encoder 208 may potentially evaluate each block with regard to each possible template partitioning, thus potentially evaluating each block up to 64 times. This tends to be undesirable in the video coding process, as computational overhead of the video encoder 208 is greatly compounded as a result.
  • example embodiments of the present disclosure provide a probabilistic geometric partitioning method.
  • FIG. 4 illustrates a flowchart of a geometric partitioning method 400 implementing probabilistic thresholds according to example embodiments of the present disclosure.
  • a video encoder obtains a source frame from a video source.
  • the video encoder may begin encoding the frame on a per-block basis in a coding order among blocks of the frame, such as a raster scan order wherein a first-decoded block is an uppermost and leftmost block of the frame, according to video encoding standards.
  • a coding order among blocks of the frame such as a raster scan order wherein a first-decoded block is an uppermost and leftmost block of the frame, according to video encoding standards.
  • Subsequent steps of the method 400 at least prior to step 414 and possibly encompassing steps 414 and 416 (as shall be described in more detail below) , may be part of the overall encoding process, according to example embodiments of the present disclosure.
  • the video encoder scans a current block of the source frame.
  • any number of blocks of the source frame may have already been scanned and encoded in manners according to AVC, HEVC, VVC, and such video coding standards implementing motion prediction, and further including steps as subsequently described.
  • the current block may be a block or sub-block having dimensions of at least 8 luma samples to each side, as described above.
  • the video encoder selects a template partitioning according to geometric partitioning with regard to the current block.
  • the selected template partitioning may be one among some number of template partitionings provided by an implementation of GEO, as described above.
  • the selected template partitioning may be any arbitrary one of the template partitionings, wherein each of the template partitionings may be uniquely identified such as by a number.
  • the video encoder may select a template partitioning by looping through each unique identification of template partitionings. For sixty-four template partitionings provided by an implementation of GEO, the loop may increment an index through values from 0 to 63.
  • the video encoder may perform a set of probabilistic search steps based on branching decision logic.
  • FIG. 5A illustrates a flowchart of probabilistic search steps according to an example embodiment of the present disclosure
  • FIGS. 5B, 5C, and 5D illustrate flowcharts of probabilistic search steps according to additional example embodiments of the present disclosure.
  • Step 406 of the method 400 is illustrated in each of these Figures as step 502. Subsequent steps of the method 400 shall be described with reference to these Figures.
  • the video encoder decides whether to skip evaluation of the template partitioning with regard to the current block based on a probabilistic metric of the template partitioning.
  • Evaluation of the template partitioning may be performed normally according to implementations of GEO, and methods and techniques thereof need not be reiterated herein; performed normally, the evaluation would result in a determination that an individual template partitioning should be applied to the current block, or should not be applied to the current block.
  • evaluation of the individual template partitioning may be skipped entirely with regard to the current block; this outcome will ensure that the individual template partitioning will not be applied to the current block, under the assumption that this is the most probabilistically correct outcome out of the two possible outcomes.
  • computational resources that would have been incurred by the evaluation are conserved.
  • the video encoder may decide, at a step 504, whether to skip evaluation of the template partitioning with regard to the current block by looking up a probability of the template partitioning, and determining whether it is higher than a probability threshold. In the case that the probability is higher than the probability threshold, the video encoder may decide to not skip evaluation of the template partitioning, and in the case that the probability is lower than the probability threshold, the video encoder may decide to skip evaluation of the template partitioning; consequences of these decisions shall be described with respect to subsequent steps.
  • a probability of an individual template partitioning may be a probabilistic value corresponding to the likelihood that the individual template partitioning applies.
  • a probability of a template partitioning may be dependent on the unique identification of the template partitioning.
  • the probability of an individual template partitioning p may be written as prob (p) .
  • the probability threshold may be written as t p , and may be, for example, an arbitrary value which has been experimentally determined from results of performing the method 400 on test image data.
  • the video encoder may decide, at a step 504, whether to skip evaluation of the template partitioning with regard to the current block by looking up a position of the template partitioning in a sorted order of template partitionings, and determining whether the position is lower than an order threshold. In the case that the position is lower than the order threshold, the video encoder may decide to not skip evaluation of the template partitioning, and in the case that the position is higher than the order threshold, the video encoder may decide to skip evaluation of the template partitioning; consequences of these decisions shall be described with respect to subsequent steps.
  • a sorted order of template partitionings may be an index which orders each template partitioning according to an implementation of GEO by a respective probability of that template partitioning. For example, a template partitioning having a highest probability may have a lowest index position (such as 0) , and a template partitioning having a lowest probability may have a highest index position. According to example embodiments of the present disclosure, since a probability of a template partitioning may be dependent on the unique identification of the template partitioning, the same may be true of positions of template partitionings in the sorted order. Thus, the position of an individual template partitioning p in a sorted order of template partitionings may be written as sort (p) .
  • the order threshold may be written as t s , and may be, for example, an arbitrary value which has been experimentally determined from results of performing the method 400 on test image data.
  • the video encoder may decide, at a step 504, whether to skip evaluation of the template partitioning with regard to the current block by looking up both a probability of the template partitioning and a position of the template partitioning, determining whether the probability threshold is higher than a probability threshold, and determining whether the position is lower than an order threshold.
  • the video encoder may decide to not skip evaluation of the template partitioning, and in the case that either the position is lower than the order threshold or the position is higher than the order threshold, the video encoder may decide to skip evaluation of the template partitioning.
  • the video encoder may decide, at a step 504, whether to skip evaluation of the template partitioning with regard to the current block by looking up both a probability of the template partitioning and a position of the template partitioning, determining whether the probability threshold is higher than a probability threshold, and determining whether the position is lower than an order threshold.
  • the video encoder may decide to not skip evaluation of the template partitioning, and in the case that both the position is lower than the order threshold and the position is higher than the order threshold, the video encoder may decide to skip evaluation of the template partitioning.
  • probabilities of template partitionings may be recorded at a granularity level of the source frame, or may be recorded at a granularity level of particular blocks therein.
  • Initial probabilities may be initialized for each template partitioning and set for a first frame of each frame type: i.e., intra-coded frame, inter-coded frame, predicted frames ( “P-frames” ) , bi-directional predicted frames ( “B- frames” ) , and the like, and updated according to steps of the method 400 thereafter.
  • initial probabilities may be initialized for each unique block dimension for blocks wherein GEO is performed, as described above.
  • probabilities of each template partitioning may be reset to initial probabilities upon detection of a scene change in a source frame from image data of previous frames of the source video.
  • the probabilities of template partitionings may be maintained as suitable probabilistic predictors of a template partitioning to be applied frame-by-frame over image data of the source video, as well as in manners which shall be further illustrated by subsequent descriptions.
  • both the probability of the template partitioning and the position of the template partitioning are probabilistic metrics of the template partitioning.
  • the video encoder executes the decision of whether to skip evaluation of the template partitioning with regard to the current block.
  • FIGS. 5A, 5B, 5C, and 5D illustrate, at a step 506, based on a decision to not skip evaluation of the template partitioning with regard to the current block, the video encoder evaluates the template partitioning with regard to the current block based on a GEO implementation. This may result in either applying the template partitioning as a partitioning of the current block, or not applying the template partitioning.
  • the video encoder further computes a cost of applying the template partitioning.
  • a cost function may be any arbitrarily defined function as known in the art, formulated based on output of encoding the block by applying the template partitioning based on implementations of GEO, where output of the cost function represents extent of deviations of each bit from image data of the video source.
  • the cost function may be defined to yield greater outputs for encoded blocks which less accurately match image data of the video source; for example, the cost function may be defined according to rate-distortion optimization.
  • the video encoder After computing the cost of applying the template partitioning, the video encoder further updates a minimal cost.
  • the minimal cost may be defined at a granularity level of the source frame or a granularity level of the current block, as described above. In the event that cost determined at this step is smaller than a minimal cost previously recorded, the cost determined at this step replaces the previously recorded minimal cost.
  • FIGS. 5A, 5B, 5C, and 5D illustrate, at a step 508, based on a decision to skip evaluation of the template partitioning with regard to the current block, the video encoder determines whether any template partitionings remain to be selected with regard to the current block (that is, template partitionings which have not yet been selected with regard to the current block in prior iterations of the method 400) . In the case that any template partitionings remain to be selected, another template partitioning is selected at step 502 (for example, by incrementing the loop index as described above with respect to step 406) . In the case that no template partitionings remain to be selected, the geometric partitioning method 400 is terminated at a step 514 which shall be subsequently described.
  • the video encoder decides whether to update the probabilistic metric of the template partitioning.
  • FIGS. 5A, 5B, 5C, and 5D illustrate, at a step 510, the video encoder determines whether the minimal cost is smaller than a cost threshold.
  • the cost threshold may be written as t c , and may be, for example, an arbitrary value which has been experimentally determined from results of performing the method 400 on test image data.
  • the video encoder determines whether any template partitionings remain to be selected with regard to the current block (that is, template partitionings which have not yet been selected with regard to the current block in prior iterations of the method 400) .
  • the video decoder decides to update the probability of the template partitioning p prob (p) ; in the cases of FIGS. 5B, 5C, and 5D, the video decoder further updates the position of the template partitioning p in the sorted order of template partitionings sort (p) , as a consequence of the sorted order potentially changing due to the probability of the template positioning being updated.
  • step 502 In the case that the minimal cost is larger than the cost threshold and any template partitionings remain to be selected, another template partitioning is selected at step 502 (for example, by incrementing the loop index as described above with respect to step 406) .
  • the video encoder updates the probabilistic metric of the template partitioning.
  • FIGS. 5A, 5B, 5C, and 5D illustrate, at a step 512, the video encoder updates prob (p) , and, in the cases of FIGS. 5B, 5C, and 5D, updates sort (p) .
  • the video encoder updates prob (p) (as well as sort (p) ) after the encoding process as described above with reference to step 402 is completed for the source frame. That is, the overall encoding process as described above with reference to step 402 may be completed prior to step 414. This is illustrated in FIGS. 5A, 5B, 5C, and 5D by the broken lines around step 512, indicating the case where step 512 is not performed within the overall encoding process.
  • the video encoder updates prob (p) (as well as sort (p) ) after applying the template partitioning as described above with reference to step 410 is completed for the current block. That is, the overall encoding process as described above with reference to step 402 may still be ongoing during step 414.
  • Updating prob (p) may be performed according to the following sub-steps:
  • unique block dimensions may include 8x8, 8x16, 16x8, 16x16, and such luma sample dimensions
  • the count of each unique template partitioning i.e., there may be 64 unique template partitionings, as described above.
  • a sum is taken over the counts of each unique template partitioning occurring for each unique block dimension. For example, given four unique block dimensions and 64 unique template partitionings, the sum is taken over 256 separate counts of template partitioning occurrence.
  • a new probability is computed for each combination of unique block dimension and unique template partitioning by dividing each count of template partitioining occurrence by the sum.
  • an existing probability i.e., an initial probability or a probability updated by a previous iteration of the method 400
  • the existing probability and the new probability may be added and averaged to derive an updated probability within a same probability range as the existing probability.
  • the existing probability may be weighted by a first weighting factor and the new probability may be weighted by a second weighting factor, and the weighted existing probability and the weighted new probability may be added to derive an updated probability within a same probability range as the existing probability (by, for example, setting the first weighting factor and the second weighting factor to sum to 1) .
  • probabilities of individual template partitionings throughout the scene may be continuously updated.
  • Characteristic template partitionings of frames of the scene may have their respective probabilities increased, and non-characteristic template partitionings of frames of the scene may have their respective probabilities decreased.
  • the video encoder may efficiently skip evaluations of non- characteristic template partitionings with regard to blocks with increasing efficiency as it encodes more blocks and more frames of the scene.
  • the video encoder completes geometric partitioning with regard to the current block.
  • FIGS. 5A, 5B, 5C, and 5D illustrate, at a step 514, the video encoder completes the geometric partitioning method 400.
  • the video encoder completes the present iteration of the geometric partitioning method 400 and then proceeds to scan the next block as described above with reference to step 404. That is, the overall encoding process as described above with reference to step 402 may still be ongoing after step 416, without terminating GEO overall.
  • FIG. 6 illustrates an example system 600 for implementing the processes and methods described above for implementing probabilistic geometric partitioning.
  • the techniques and mechanisms described herein may be implemented by multiple instances of the system 600 as well as by any other computing device, system, and/or environment.
  • the system 600 shown in FIG. 6 is only one example of a system and is not intended to suggest any limitation as to the scope of use or functionality of any computing device utilized to perform the processes and/or procedures described above.
  • the system 600 may include one or more processors 602 and system memory 604 communicatively coupled to the processor (s) 602.
  • the processor (s) 602 may execute one or more modules and/or processes to cause the processor (s) 602 to perform a variety of functions.
  • the processor (s) 602 may include a central processing unit ( “CPU” ) , a graphics processing unit ( “GPU” ) , both CPU and GPU, or other processing units or components known in the art. Additionally, each of the processor (s) 602 may possess its own local memory, which also may store program modules, program data, and/or one or more operating systems.
  • system memory 604 may be volatile, such as RAM, non-volatile, such as ROM, flash memory, miniature hard drive, memory card, and the like, or some combination thereof.
  • the system memory 604 may include one or more computer-executable modules 606 that are executable by the processor (s) 602.
  • the modules 606 may include, but are not limited to, an encoder module 608.
  • the encoder module 608 further includes a source frame obtaining submodule 610, a block scanning submodule 612, a template partitioning selecting submodule 614, a skip deciding submodule 616, a skip decision executing submodule 618, an update deciding submodule 620, an updating submodule 622, and a geometric partition completing submodule 624.
  • the encoder module 608 may be configured to perform motion prediction coding upon frames from a video source by any of the algorithms and processes described herein, including the functionality of each submodule described herein.
  • the source frame obtaining submodule 610 may be configured to obtain a source frame from a video source, as described above with reference to FIG. 4.
  • the block scanning submodule 612 may be configured to scan a current block of the source frame, as described above with reference to FIG. 4.
  • the template partitioning selecting submodule 614 may be configured to select a template partitioning with regard to the current block, as described above with reference to FIG. 4.
  • the skip deciding submodule 616 may be configured to decide whether to skip evaluation of the template partitioning with regard to the current block based on a probabilistic metric of the template partitioning, as described above with reference to FIG. 4.
  • the skip decision executing submodule 618 may be configured to execute the decision of whether to skip evaluation of the template partitioning with regard to the current block, as described above with reference to FIG. 4.
  • the update deciding submodule 620 may be configured to decide whether to update the probabilistic metric of the template partitioning, as described above with reference to FIG. 4.
  • the updating submodule 622 may be configured to update the probabilistic metric of the template partitioning, as described above with reference to FIG. 4.
  • the geometric partition completing submodule 624 may be configured to complete geometric partitioning with regard to the current block, as described above with reference to FIG. 4.
  • the system 600 may additionally include an input/output (I/O) interface 640 for receiving video source data and bitstream data, and for outputting decoded frames into a reference frame buffer and/or a display buffer.
  • the system 600 may also include a communication module 650 allowing the system 600 to communicate with other devices (not shown) over a network (not shown) .
  • the network may include the Internet, wired media such as a wired network or direct-wired connections, and wireless media such as acoustic, radio frequency ( “RF” ) , infrared, and other wireless media.
  • RF radio frequency
  • Computer-readable instructions include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like.
  • Computer-readable instructions can be implemented on various system configurations, including single-processor or multiprocessor systems, minicomputers, mainframe computers, personal computers, hand-held computing devices, microprocessor-based, programmable consumer electronics, combinations thereof, and the like.
  • the computer-readable storage media may include volatile memory (such as random-access memory ( “RAM” ) ) and/or non-volatile memory (such as read-only memory ( “ROM” ) , flash memory, etc. ) .
  • volatile memory such as random-access memory ( “RAM” )
  • non-volatile memory such as read-only memory ( “ROM” ) , flash memory, etc.
  • the computer-readable storage media may also include additional removable storage and/or non-removable storage including, but not limited to, flash memory, magnetic storage, optical storage, and/or tape storage that may provide non-volatile storage of computer-readable instructions, data structures, program modules, and the like.
  • a non-transient computer-readable storage medium is an example of computer-readable media.
  • Computer-readable media includes at least two types of computer-readable media, namely computer-readable storage media and communications media.
  • Computer-readable storage media includes volatile and non-volatile, removable and non-removable media implemented in any process or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.
  • Computer-readable storage media includes, but is not limited to, phase change memory ( “PRAM” ) , static random-access memory ( “SRAM” ) , dynamic random-access memory ( “DRAM” ) , other types of random-access memory ( “RAM” ) , read-only memory ( “ROM” ) , electrically erasable programmable read-only memory ( “EEPROM” ) , flash memory or other memory technology, compact disk read-only memory ( “CD-ROM” ) , digital versatile disks ( “DVD” ) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device.
  • PRAM phase change memory
  • SRAM static random-access memory
  • DRAM dynamic random-access memory
  • RAM random-access memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • flash memory or other memory technology compact disk read-only memory ( “
  • communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanism.
  • a computer-readable storage medium employed herein shall not be interpreted as a transitory signal itself, such as a radio wave or other free-propagating electromagnetic wave, electromagnetic waves propagating through a waveguide or other transmission medium (such as light pulses through a fiber optic cable) , or electrical signals propagating through a wire.
  • the computer-readable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, may perform operations described above with reference to FIGS. 1A-5.
  • computer-readable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types.
  • the order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
  • the present disclosure provides probabilistically applying partitioning to individual blocks of frames based on implementations of a geometric partitioning during operation of a video encoder according to video coding standards. Evaluation of some number of template partitionings may be skipped entirely with regard to a block, under assumptions of a probabilistically correct outcome, conserving computational resources that would have been incurred by the evaluation.
  • the methods and systems described herein include a video encoder selecting a template partitioning with regard to a current block; deciding whether to skip evaluation of the template partitioning with regard to the current block based on a probabilistic metric of the template partitioning; executing the decision; deciding whether to update the probabilistic metric of the template partitioning; and updating the probabilistic metric.
  • a method comprising: selecting a template partitioning according to geometric partitioning with regard to a current block of a source frame; deciding whether to skip evaluation of the template partitioning with regard to the current block based on a probabilistic metric of the template partitioning; executing the decision of whether to skip evaluation of the template partitioning with regard to the current block; deciding whether to update the probabilistic metric of the template partitioning; and updating the probabilistic metric of the template partitioning.
  • the probabilistic metric comprises at least one of a probability of the template partitioning and a position of the template partitioning in a sorted order of template partitionings.
  • updating the probabilistic metric of the template partitioning comprises updating the probability of the template partitioning and further updating the position of the template partitioning in the sorted order of template partitionings.
  • a system comprising: one or more processors; and memory communicatively coupled to the one or more processors, the memory storing computer-executable modules executable by the one or more processors that, when executed by the one or more processors, perform associated operations, the computer-executable modules including: an encoder module comprising: a template partitioning selecting submodule configured to select a template partitioning according to geometric partitioning with regard to a current block of a source frame; a skip deciding submodule configured to decide whether to skip evaluation of the template partitioning with regard to the current block based on a probabilistic metric of the template partitioning; a skip decision executing submodule configured to execute the decision of whether to skip evaluation of the template partitioning with regard to the current block; an update deciding submodule configured to decide whether to update the probabilistic metric of the template partitioning; and an updating submodule configured to update the probabilistic metric of the template partitioning.
  • an encoder module comprising: a template partitioning selecting submodule configured to select a template partitioning
  • the probabilistic metric comprises at least one of a probability of the template partitioning and a position of the template partitioning in a sorted order of template partitionings.
  • N The system as paragraph M recites, wherein the updating submodule is configured to update the probabilistic metric of the template partitioning by updating the probability of the template partitioning and further updating the position of the template partitioning in the sorted order of template partitionings.
  • a computer-readable storage medium storing computer-readable instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising: selecting a template partitioning according to geometric partitioning with regard to a current block of a source frame; deciding whether to skip evaluation of the template partitioning with regard to the current block based on a probabilistic metric of the template partitioning; executing the decision of whether to skip evaluation of the template partitioning with regard to the current block; deciding whether to update the probabilistic metric of the template partitioning; and updating the probabilistic metric of the template partitioning.
  • deciding whether to skip evaluation of the template partitioning with regard to the current block comprises looking up a probability of the template partitioning and determining whether the probability is higher than a probability threshold.
  • deciding whether to skip evaluation of the template partitioning with regard to the current block comprises looking up a position of the template partitioning in a sorted order of template partitionings and determining whether the position is lower than an order threshold.
  • the computer-readable storage medium as paragraph O recites, wherein the probabilistic metric comprises at least one of a probability of the template partitioning and a position of the template partitioning in a sorted order of template partitionings.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Systèmes et procédés permettant d'appliquer de manière probabiliste un partitionnement à des blocs individuels de trames sur la base d'implémentations d'un partitionnement géométrique pendant le fonctionnement d'un codeur vidéo selon des normes de codage vidéo. L'évaluation d'un certain nombre de partitionnements de modèle peut être entièrement ignorée pour un bloc, dans l'hypothèse d'un résultat probabilistiquement correct, ce qui permet de conserver des ressources de calcul qui auraient été nécessaires pour l'évaluation. Les procédés et systèmes de l'invention comprennent un codeur vidéo sélectionnant un partitionnement de modèle pour un bloc actuel ; décidant s'il faut ignorer l'évaluation du partitionnement de modèle pour le bloc actuel sur la base d'une métrique probabiliste du partitionnement de modèle ; exécutant la décision ; décidant s'il faut mettre à jour la métrique probabiliste du partitionnement de modèle ; et mettant à jour la métrique probabiliste.
PCT/CN2020/096988 2020-06-19 2020-06-19 Partitionnement géométrique probabiliste en codage de vidéo WO2021253373A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/096988 WO2021253373A1 (fr) 2020-06-19 2020-06-19 Partitionnement géométrique probabiliste en codage de vidéo

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/096988 WO2021253373A1 (fr) 2020-06-19 2020-06-19 Partitionnement géométrique probabiliste en codage de vidéo

Publications (1)

Publication Number Publication Date
WO2021253373A1 true WO2021253373A1 (fr) 2021-12-23

Family

ID=79268917

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/096988 WO2021253373A1 (fr) 2020-06-19 2020-06-19 Partitionnement géométrique probabiliste en codage de vidéo

Country Status (1)

Country Link
WO (1) WO2021253373A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114422781A (zh) * 2022-03-29 2022-04-29 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质
WO2023132941A1 (fr) * 2022-01-04 2023-07-13 Qualcomm Incorporated Mode de partition géométrique dans un codage vidéo
WO2023158765A1 (fr) * 2022-02-16 2023-08-24 Beijing Dajia Internet Information Technology Co., Ltd. Procédés et dispositifs de réordonnancement de modes de division de mode de partitionnement géométrique avec un ordre de modes prédéfini

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106340023A (zh) * 2016-08-22 2017-01-18 腾讯科技（深圳）有限公司 图像分割的方法和装置
US20170353730A1 (en) * 2016-06-03 2017-12-07 Mediatek Inc. Method and Apparatus for Template-Based Intra Prediction in Image and Video Coding
WO2019185815A1 (fr) * 2018-03-29 2019-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Concepts de partitionnement pour codage d'image à base de blocs
CN110809888A (zh) * 2017-06-30 2020-02-18 华为技术有限公司 用于处理视频序列帧的编码器、解码器、计算机程序和计算机程序产品

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170353730A1 (en) * 2016-06-03 2017-12-07 Mediatek Inc. Method and Apparatus for Template-Based Intra Prediction in Image and Video Coding
CN106340023A (zh) * 2016-08-22 2017-01-18 腾讯科技（深圳）有限公司 图像分割的方法和装置
CN110809888A (zh) * 2017-06-30 2020-02-18 华为技术有限公司 用于处理视频序列帧的编码器、解码器、计算机程序和计算机程序产品
WO2019185815A1 (fr) * 2018-03-29 2019-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Concepts de partitionnement pour codage d'image à base de blocs

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
J. CHEN, Y. YE, S. KIM: "Algorithm description for Versatile Video Coding and Test Model 9 (VTM 9)", 130. MPEG MEETING; 20200420 - 20200424; ALPBACH; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), no. m53984 ; JVET-R2002, 15 June 2020 (2020-06-15), XP030289596 *
R. YU (ERICSSON), D. LIU, C. HOLLMANN, J. STRöM (ERICSSON): "Modifications of motion storage in geometric partition mode", 130. MPEG MEETING; 20200420 - 20200424; ALPBACH; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11), no. m53190 ; JVET-R0213, 3 April 2020 (2020-04-03), XP030286185 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023132941A1 (fr) * 2022-01-04 2023-07-13 Qualcomm Incorporated Mode de partition géométrique dans un codage vidéo
WO2023158765A1 (fr) * 2022-02-16 2023-08-24 Beijing Dajia Internet Information Technology Co., Ltd. Procédés et dispositifs de réordonnancement de modes de division de mode de partitionnement géométrique avec un ordre de modes prédéfini
CN114422781A (zh) * 2022-03-29 2022-04-29 深圳传音控股股份有限公司 图像处理方法、智能终端及存储介质

Similar Documents

Publication Publication Date Title
US11368675B2 (en) Method and device for encoding and decoding intra-frame prediction
CN109314785B (zh) 用于导出运动预测信息的方法和装置
US10009615B2 (en) Method and apparatus for vector encoding in video coding and decoding
WO2021253373A1 (fr) Partitionnement géométrique probabiliste en codage de vidéo
CN104244007B (zh) 一种图像编码方法和装置及解码方法和装置
US20210266547A1 (en) Encoder, decoder, computer program and computer program product for processing a frame of a video sequence
US11663476B2 (en) Method and device for providing compression and transmission of training parameters in distributed processing environment
CN116248864A (zh) 利用包含追加区域的分割单位的影像解码方法以及装置
EP3869807A1 (fr) Procédé et dispositif de codage et de décodage d'image
TWI836542B (zh) 用以編碼及解碼視訊取樣之轉換區塊的方法、設備和非暫態電腦可讀取儲存媒體
US20230297833A1 (en) Method and device for providing compression and transmission of training parameters in distributed processing environment
US11381809B2 (en) Intra prediction encoding/decoding method and apparatus for chrominance components
US20230362401A1 (en) Image encoding/decoding method and apparatus
US20200137387A1 (en) Encoder, decoder, computer program and computer program product for processing a frame of a video sequence
US11558608B2 (en) On split prediction
US20210127128A1 (en) Method and Apparatus for Motion Vector Prediction
KR20190042732A (ko) 인트라 예측 모드 기반 영상 처리 방법 및 이를 위한 장치
WO2020243100A1 (fr) Procédés et appareil d'amélioration d'estimation du mouvement dans un codage vidéo
CN114845101A (zh) 用于视频编解码的系统和方法
CN116868571A (zh) 对于帧间预测的改进的局部光照补偿
CN116636211A (zh) 利用块合并编码视频的方法和装置
KR20210046804A (ko) 영상 부호화/복호화 방법 및 장치
WO2024108391A1 (fr) Procédé et appareil de codage vidéo, procédé et appareil de décodage vidéo, et dispositifs, système et support d'enregistrement
WO2024077553A1 (fr) Procédé et appareil de codage vidéo, procédé et appareil de décodage vidéo, dispositif, système, et support de stockage
US20240040133A1 (en) Video coding using multi-model linear model

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20940796

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20940796

Country of ref document: EP

Kind code of ref document: A1