TW202420819A - Prediction generation with out-of-boundary check in video coding - Google Patents

Abstract

A method of coding video pictures using predictive coding with out-of-bound (OOB) checks is provided. A video coder receives data to be encoded or decoded as a current block of a current picture of a video. The video coder identifies a first reference block in a first reference picture based on a first block vector of the current block and a second reference block in a second reference picture based on a second block vector of the current block. The video coder performs out-of-bound (OOB) check for the first and second reference blocks. The video coder generates a predictor for the current block based on the first and second reference blocks and based on the OOB check, such that the use of bi-directional motion compensation can be constrained. The video coder encodes or decodes the current block by using the generated predictor.

Description

Prediction generation with out-of-bounds checking in video encoding and decoding

The present invention relates to video coding and decoding. More specifically, the present invention relates to a method for coding and decoding pixel blocks by inter-frame and intra-frame prediction.

Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

High Efficiency Video Coding (HEVC) is an international video coding standard developed by the Joint Collaboration on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion-compensated DCT-like transform coding architecture. The basic unit of compression, called a coding unit (CU), is a 2Nx2N square block of pixels, and each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs).

Versatile Video Codec (VVC) is the latest international video codec standard developed by ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 Joint Video Experts Group (JVET). The input video signal is predicted based on a reconstructed signal derived from a coded picture region. The prediction residual signal is processed by a block transform. The transform coefficients are quantized and entropy coded along with other auxiliary information in the bitstream. The reconstructed signal is generated based on the prediction signal and the reconstructed residual signal after inverse transforming the dequantized transform coefficients. The reconstructed signal is further processed by loop filtering to remove coding artifacts. The decoded pictures are stored in a frame buffer and used to predict future pictures in the input video signal.

In VVC, the coded picture is divided into non-overlapping square block areas represented by associated codec tree units (CTUs). The leaf nodes of the codec tree correspond to codec units (CUs). The coded picture can be represented by a set of slices, each slice containing an integer number of CTUs. Each CTU in a slice is processed in raster scan order. Bi-predictive (B) slices can be decoded using intra-frame prediction or inter-frame prediction, with up to two motion vectors and reference indices to predict the sample values of each block. Predictive (P) slices are decoded using intra-frame prediction or inter-frame prediction with up to one motion vector and reference index to predict the sample values of each block. Only Intra (I) slices are decoded using intra prediction.

A CTU can be divided into one or more non-overlapping coding units (CUs) using a quadtree (QT) with a nested multi-type-tree (MTT) structure to accommodate various local motion and texture characteristics. A CU can be further split into smaller CUs using one of five partitioning types: quadtree partitioning, vertical binary tree partitioning, horizontal binary tree partitioning, vertical center-side tritree partitioning, and horizontal center-side tritree partitioning.

Each CU contains one or more prediction units (PUs). The prediction unit together with the associated CU syntax serves as the basic unit for sending prediction information. The specified prediction process is used to predict the values of the relevant pixel samples within the PU. Each CU can contain one or more transform units (TUs) for representing prediction residue blocks. A transform unit (TU) consists of a transform block (TB) of luma samples and two corresponding transform blocks of chroma samples, each TB corresponding to a residue block of samples from one color component. An integer transform is applied to the transform block. The level value of the quantization coefficient is entropy encoded and decoded in the bitstream along with other auxiliary information. The terms codec tree block (CTB), codec block (CB), prediction block (PB), and transform block (TB) are defined to specify a 2-D sample array of one color component associated with a CTU, CU, PU, and TU, respectively. Thus, a CTU consists of one luma CTB, two chroma CTBs, and related syntax elements. Similar relationships are valid for CU, PU, and TU.

For each inter-frame prediction CU, motion parameters consisting of motion vectors, reference picture index and reference picture list usage index, as well as additional information are used for inter-frame prediction sample generation. Motion parameters can be signaled explicitly or implicitly. When the CU is encoded or decoded in skip mode, the CU is associated with one PU and has no significant residual coefficients, no coded motion vector increments or reference picture indices. Specifying the merge mode, the motion parameters of the current CU are obtained from neighboring CUs, including spatial and temporal candidates, as well as the additional scheduling introduced in VVC. The merge mode can be applied to any inter-frame prediction CU. An alternative to merge mode is explicit transmission of motion parameters, where motion vectors, the corresponding reference picture index for each reference picture list and the reference picture list usage flag, along with other required information, are explicitly sent per CU.

The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technologies described herein. Selected but not all implementations are further described in the detailed description below. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

Some embodiments of the present disclosure provide a video encoding and decoding method, including: receiving data of a current block of a current picture to be encoded or decoded into a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied to at least one of the horizontal and vertical directions; and encoding or decoding the current block using the generated predictor.

Some embodiments of the present disclosure provide an electronic device, including: a video encoder circuit configured to perform operations, including: receiving data of a current block of a current picture to be encoded or decoded as a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied to at least one of the horizontal and vertical directions; and encoding or decoding the current block using the generated predictor.

Some embodiments of the present disclosure provide a video decoding method, including: receiving data of a current block of a current picture to be decoded as a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied to at least one of the horizontal and vertical directions; and reconstructing the current block using the generated predictor.

The encoding and decoding method of the present invention is that the relevant device performs cross-border detection to generate prediction samples.

In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. Any variations, derivations, and/or extensions based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, procedures, components, and/or circuits associated with one or more example implementations disclosed herein may be described at a relatively high level without detail to avoid unnecessarily obscuring aspects of the teachings of the present disclosure. I. Motion Compensation Out-of-Boundary Check

An inter-frame predicted CU may have reference blocks that are partially or completely outside the reference picture. Figure 1 illustrates bidirectional prediction using out-of-boundary (OOB) reference blocks. In the figure, bidirectional motion compensation is performed to generate an inter-frame prediction block (predictor) for the current block 105 in the current picture 100. List 0 reference block 120 partially exceeds the reference block picture 110. List 1 reference block 121 is completely located inside the reference picture 111. When the video codec executes the prediction generation process 130, the OOB portion 140 of the reference block 120 is filled with repetitive padding samples. The prediction generation process 130 generates a prediction block 135 for the current block 105 using the reference blocks 120 and 121. Since the OOB portion 140 of the reference block 120 is filled with repeated samples derived from boundary samples within the reference picture 110, the portion of the motion compensation block 135 corresponding to the OOB portion 140 may provide lower prediction efficiency.

In some embodiments, a unidirectional motion compensation sample is considered OOB when one of its reference samples is outside the reference picture for more than half of the samples. For each prediction sample in a bidirectional motion compensation block, when one of its unidirectional prediction samples is OOB and the other is non-OOB, the corresponding bidirectional sample is set equal to the non-OOB sample instead of using the average of the OOB and non-OOB samples.

In some embodiments, when more than one prediction block is combined, the OOB prediction samples are discarded and only non-OOB predictors are used to generate the final predictor. Specifically, let Pos_xi _,j and Pos_y _,j represent the x and y positions of a prediction sample (i, j) in the current block, MV ^LX_xi _,j and MV ^LX_y _,j (LX = L0 or L1) represent the MV of the current block; _PosLeftBdry , _PosRightBdry , _PosTopBdry , and _{PosBottomBdry} are the positions of the four boundaries of the image. A forecast sample is considered OOB when at least one of the following conditions is met: (Pos_xi _,j + ^MVLX_xi _,j ) > ( _PosRightBdry + half_pixel ), (Pos_xi _,j + ^MVLX_xi _,j ) < ( _PosLeftBdry + half_pixel ), (Pos_yi _,j + ^MVLX_yi _,j ) > ( _{PosBottomBdry} + half_pixel ), (Pos_yi _,j + ^MVLX_yi _,j ) > ( _PosTopBdry + half_pixel )

Where half_pixel is equal to 8, indicating the half-pixel sample distance in 1/16 pixel sample accuracy. After checking the OOB status of each sample, the final prediction sample of a bidirectional block is generated as follows: If ^PL0 _i,j is OOB and ^PL1 _i,j is non-OOB, then P ^final _i,j = ^PL1 _i,j Otherwise If ^PL0 _i,j is non-OOB and ^PL1 _i,j is OOB, then P ^final _i,j = ^PL0 _i,j Otherwise P ^final _i,j = ( ^PL0 _i,j + ^PL1 _i,j + 1) >> 1 II. Block Encoding and Decoding Prediction Tool

a. Horizontal orbital motion compensation ( Horizontal Wrap Around Motion Compensation ）

Horizontal surround motion compensation is a 360-degree specific codec tool designed to improve the visual quality of reconstructed 360-degree videos in the equirectangular projection (ERP) format. In traditional motion compensation, when a motion vector refers to samples outside the picture boundaries of a reference picture, repeated padding is applied to derive the value of the out-of-bounds samples by copying from the nearest neighbor on the corresponding picture boundary. For 360-degree videos, this approach to repeated padding is inappropriate and may cause visual artifacts called "seam artifacts" in the reconstructed viewport video. Since 360-degree video is captured on a sphere and has essentially no "borders", reference samples that are beyond the borders of the reference image in the projection domain can always be obtained from neighboring samples in the spherical domain.

For general projection formats, it may be difficult to derive the corresponding neighboring samples in the spherical domain, because doing so involves 2D to 3D and 3D to 2D coordinate conversions, as well as sample interpolation of fractional sample positions. For the left and right boundaries of the ERP projection format, this problem is much simpler, because the spherical neighbors outside the left image boundary can be obtained from the samples inside the right image boundary, and vice versa. Therefore, horizontal surround motion compensation can be used to improve the visual quality of 360-degree videos encoded and decoded in the ERP projection format.

In some embodiments, horizontal surround motion compensation can be combined with a (non-canonical) padding method commonly used in 360-degree video codecs. This can be achieved by signaling a high-level syntax element to indicate a surround offset, which can be set to the padding width applied to the ERP picture. This syntax can be used to adjust the position of the horizontal surround accordingly. This syntax is not affected by the specific amount of padding on the left and right picture borders, and therefore naturally supports asymmetric padding of ERP pictures, i.e. when the left and right padding are different. Horizontal surround motion compensation provides more meaningful information for motion compensation when the reference samples are outside the left and right borders of the reference picture.

b. Multi-hypothesis frame prediction ( Multi-Hypothesis Prediction , abbreviated as MHP ）

In the multi-hypothesis inter-frame prediction mode (MHP), in addition to the traditional bidirectional prediction signal, one or more additional motion compensated prediction signals are signaled. The final overall prediction signal is obtained by sample weighted superposition. Using the bidirectional prediction signal _pbi and the first additional inter-frame prediction signal/hypothesis _h3 , the resulting prediction signal _p3 is obtained according to the following formula: _p3 = (1–α) _pbi + _αh3

The weighting factor α (or the mixing ratio) is specified by the syntax element add_hyp_weight_idx in the codec video as 1/4 or -1/8, etc.

FIG. 2 conceptually illustrates a multi-hypothesis prediction (MHP) for encoding and decoding a current block 220. The current block 220 is in a current picture 200. The current block has bidirectional prediction motion vectors MV0 and MV1 used to obtain predictors 230 (in reference picture 210) and 231 (in reference picture 211), respectively. The bidirectional prediction predictors are used to generate a bidirectional prediction signal _pbi . In addition to the bidirectional prediction, the current block is also encoded and decoded using MHP prediction. The MHP prediction is based on additional hypotheses selected from a plurality of MHP candidates A, B, C and D, which locate predictors or prediction signals 241, 242, 243 and 244 in different reference pictures, respectively. In the example, hypothesis C is selected to generate the additional prediction signal h ₃ , ie, the prediction signal 243 .

In some embodiments, more than one additional prediction signal may be used. The resulting overall prediction signal is iteratively accumulated with each additional prediction signal. For example, a second additional hypothesis h ₄ may be added to the prediction according to the following condition: p ₄ = (1 -α) p ₃ +α h ₄

Or more generally: p _n+1 = (1 -α _n+1 ) p _n +α _n+1 h _n+1

The resulting prediction signal pn ₊₁ is obtained as a mixture of the last _pn (the current accumulated prediction) and the latest additional prediction signal hn ₊₁ . The first existing prediction of the current block is _p0 . For example, if MHP is applied to merge candidates, _p0 is indicated by the existing merge index. The mixture of _hn+1 and pn ₊₁ is a weighted sum based on a mixing ratio or a weighting factor α. In some embodiments, for each additional hypothesis of the prediction, a weighting index can be signaled/parsed to indicate the weighting factor.

c. Combined inter-frame and intra-frame prediction （ Combined inter and intra prediction , abbreviated as CIIP)

Combined inter- and intra-frame prediction (CIIP) combines an inter-frame prediction signal with an intra-frame prediction signal. The _inter -frame prediction signal Pinter in the CIIP mode is derived using the same inter-frame prediction process applied to the conventional merge mode; while the _intra -frame prediction signal Pintra is derived after the conventional intra-frame prediction process using the planar mode or one or more intra-frame prediction modes derived from a predefined mechanism. For example, in the following, the predefined mechanism is based on the neighboring reference area (template) of the current block. The intra-frame prediction mode of the CU is implicitly derived from the neighboring templates at the encoder and decoder, instead of being sent to the decoder as exact intra-frame prediction mode bits. The prediction samples of the template are generated using the reference samples of the template for each candidate mode. The cost is calculated as the SATD between the predicted samples and the reconstructed samples of the template. The intra prediction mode with the minimum cost and/or some intra prediction modes with smaller costs are selected and used for intra prediction of the CU. The candidate modes can be all MPMs and/or any subset of MPMs, such as the 67 intra prediction modes in VVC or extended to 131 intra prediction modes. The intra and inter prediction signals are combined using a weighted average, where the weight values are calculated based on the coding and decoding modes of the top and left neighboring blocks. The CIIP prediction P _CIIP is formed as follows: (wt is the weight value) P _CIIP = ((4 – wt ) * P _inter + wt * P _intra + 2) >> 2

In some embodiments, when a CU is encoded or decoded in merge mode, if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), and if both the CU width and the CU height are less than 128 luma samples, an additional flag may be signaled to indicate whether the CIIP mode applies to the current CU.

d. Geometric partitioning mode ( Geometric partitioning Mode , GPM ）

Geometric partition mode is a coding tool for inter-frame prediction that splits the block currently being encoded into two or more parts. The partition mode of each GPM partition or GPM split is characterized by the distance-angle pairing that defines the bisecting line or segmenting line. The geometric partition mode (GPM) is sent using the CU-level flag as a merge mode. Other merge modes include normal merge mode, MMVD mode, CIIP mode, and sub-block merge mode. For each possible CU size w×h= ^2m × ²ⁿ (where m,n∈{3⋯6}, excluding 8x64 and 64x8), the geometric partition mode supports a total of 64 partitions.

FIG. 3 illustrates the partitioning of a CU by a geometric partitioning mode (GPM). The figure shows an example of a GPM split grouped by the same angle. As shown in the figure, when GPM is used, the CU is divided into at least two parts by a geometrically positioned straight line. The position of the partition line is mathematically derived from the angle and offset parameters of the specific partition.

Each partition in a CU formed by the partition pattern of GPM uses its own motion (vector) for inter-frame prediction. In some embodiments, each partition only allows unidirectional prediction, i.e., each part has one motion vector and one reference index. Similar to traditional bidirectional prediction, unidirectional prediction motion constraints are applied to ensure that only two motion compensation predictions are performed for each CU.

If GPM is used for the current CU, a geometric partition index indicating the partition mode (angle and offset) of the geometric partition and two merge indices (one for each partition) are further sent. Each of the at least two partitions created by the geometric partitioning according to the partitions can be assigned a merge index to select candidates from a unidirectional prediction candidate list (also called a GPM candidate list). Therefore, the merge index pair of the two partitions selects a pair of merge candidates. The maximum number of candidates in the GPM candidate list can be explicitly sent in the SPS to specify the syntactic binarization of the GPM merge index. After predicting each of the at least two partitions, the sample values along the edges of the geometric partitions are adjusted using a hybrid process with adaptive weights. This is the prediction signal for the entire CU, and the transform and quantization processing will be applied to the entire CU as in other prediction modes. Then the motion field of the CU predicted by GPM is stored.

The unidirectional prediction candidate list (GPM candidate list) for the GPM partition can be directly derived from the merge candidate list of the current CU. Figure 4 shows an exemplary unidirectional prediction candidate list 400 for the GPM partition and the selection of the unidirectional prediction MV for the GPM. The GPM candidate list 400 is constructed in an even-odd manner, with only unidirectional prediction candidates alternating between L0MV and L1MV. Set n to the index of the unidirectional prediction motion in the unidirectional prediction candidate list of the GPM. The LX (i.e., L0 or L1) motion vector of the nth extended merge candidate, where X is equal to the parity of n, is used as the nth unidirectional prediction motion vector of the GPM. (These motion vectors are marked with an “x” in the figure.) In the absence of a corresponding LX motion vector for the nth extended merge candidate, the L(1-X) motion vector of the same candidate is used as the unidirectional prediction motion vector of the GPM.

As mentioned before, the sample values along the edges of geometric partitions are adjusted using a blending process with adaptive weights. Specifically, after predicting each part of the geometric partition using its own motion, blending is applied to at least two prediction signals to derive samples around the edges of the geometric partitions. The blending weight for each position of the CU is derived based on the distance between the corresponding position and the partition edge. The distance from a position (x, y) to the partition edge is derived as follows:

where i , j are the angle and offset indices of the geometry partition, which depend on the geometry partition index sent. The signs of ρx _,j and ρy _,j depend on the angle index i. The weights for each part of the geometry partition are derived as follows:

The variable partIdx depends on the angle index i . FIG. 5 shows an example partition edge blending process for the GPM of CU 500. In the figure, the blending weights are generated based on the initial blending weights _w0 .

As described above, the motion field of the CU predicted using GPM is stored. Specifically, Mv1 from the first part of the geometric partition, Mv2 from the second part of the geometric partition, and the combination Mv of Mv1 and Mv2 are stored in the motion field of the CU decoded by GPM. The type of stored motion vectors for each individual position in the motion field is determined as:

where motionIdx is equal to d(4x+2,4y+2), which is recalculated from equation (2). partIdx depends on the angle index i . If sType is equal to 0 or 1, then Mv0 or Mv1 is stored in the corresponding motion field, otherwise if sType is equal to 2, then Mv from the combination of Mv0 and Mv2 is stored. The combined Mv is generated using the following process: (i) if Mv1 and Mv2 come from different reference picture lists (one from L0 and the other from L1), then Mv1 and Mv2 are simply combined to form a bidirectional predicted motion vector; (ii) otherwise, if Mv1 and Mv2 come from the same list, only the unidirectional predicted motion Mv2 is stored.

e. Template Matching (Template Matching , abbreviated as TM)

Template matching (TM) is a decoder-side MV inference method used to refine the motion information of the current CU by finding the closest match between the template of the current CU in the current picture (e.g., the top and/or left neighboring blocks of the current CU) and a set of pixels in the reference picture (i.e., the same size as the template).

FIG. 6 conceptually illustrates performing template matching based on a search area around an initial motion vector (MV). As shown, for a current CU 605 in a current picture 600, the video codec searches a reference picture or frame 601 within a [-8, +8] pixel search range around an initial MV 610 for a better or refined MV 611. The initial MV 610 identifies an initial reference template 630. The refined MV 611 identifies a refined reference template 631. The search is based on minimizing the difference (or cost) between a current template 620 and a refined reference template 631 that are adjacent to the current block 605. Template matching can be performed using a search step size determined based on an adaptive motion vector resolution (AMVR) mode. The template matching process can be cascaded with the bilateral matching process in merge mode.

In the Advanced Motion Vector Prediction (AMVP) mode, MVP candidates are determined based on template matching error to select the MVP candidate with the smallest difference between the current block template and the reference block template, and then TM is performed for MV refinement only on this specific MVP candidate. The TM process refines this MVP candidate by using an iterative diamond search starting with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode) within the [–8,+8] pixel search range. The AMVP candidate can be further refined by using a cross search with full-pixel MVD accuracy (or 4 pixels for 4-pixel AMVR mode), and then using half-pixels and quarter-pixels in turn according to the AMVR mode search pattern.

Adaptive Reordering of Merge Candidates with Template Matching (ARMC-TM) is a method for reordering merge candidates based on template matching (TM) cost, where communication efficiency is improved by sorting the merge candidates in ascending order of TM cost. For ARMC-TM or TM merging mode, the merge candidates are reordered before the refinement process. The template matching cost of the merge candidates can be measured by the sum of absolute differences (SAD) between the samples of the current template 620 of the current block and their corresponding reference samples in the reference template 621.

In some embodiments, after constructing the merge candidate list, the merge candidates are divided into several subgroups. For the normal merge mode and the TM merge mode, the subgroup size is set to 5. For the affine merge mode, the subgroup size is set to 3. The merge candidates in each subgroup are reordered in ascending order according to the cost value based on template matching. In some embodiments, the merge candidates in the last but not the first subgroup are not reordered.

For sub-block-based merge candidates with a sub-block size equal to Wsub×Hsub, the template above may include several sub-templates of size Wsub×1, and the template on the left includes several sub-templates of size 1×Hsub. In some embodiments, the motion information of the sub-blocks in the first row and first column of the current block is used to derive reference samples for each sub-template.

FIG. 7 conceptually illustrates a current block 700 with sub-block motion. The current block 700 is encoded using the motion information of the sub-blocks (sub-blocks A-G) in the first row and first column of the current block. The motion information of sub-blocks A-G is used to identify reference sub-blocks A'-G' in a reference picture. The current block 700 has a neighboring template 710, which includes sub-templates 711-717 that are adjacent to sub-blocks A-G above and to the left of the current block 700, respectively. The reference sub-blocks A'-G' have corresponding neighboring reference sub-blocks 721-727 in the reference picture. The TM cost of the motion information of different sub-blocks can be calculated by matching the sub-templates 711-717 with the corresponding sub-templates 721-727.

f. Decoder motion vector refinement (DecoderMotionVectorRefinement , abbreviated as DMVR)

To increase the accuracy of the MV in merge mode, decoder-side motion vector refinement based on bilateral-matching (BM) can be applied to refine the MV. In the bidirectional prediction operation, the refined MV is searched around the initial MV in the reference picture list L0 and the reference picture list L1. The BM method calculates the distortion between two candidate blocks in the reference picture list L0 and the list L1. The MV candidate with the lowest SAD becomes the refined MV and is used to generate the bidirectional prediction signal.

In some embodiments, if the selected merge candidate satisfies the DMVR condition, a multi-pass decoder-side motion vector refinement (MP-DMVR) method is applied in a conventional merge mode. In the first pass, bilateral matching (BM) is applied to the codec block. In the second pass, BM is applied to each 16x16 sub-block within the codec block. In the third pass, the MV in each 8x8 sub-block is refined by applying bidirectional optical flow (BDOF). BM refines a pair of motion vectors MV0 and MV1 under the constraint that the motion vector difference MVD0 (i.e., MV0'-MV0) has exactly the opposite sign to the motion vector difference MVD1 (i.e., MV1'-MV1).

FIG. 8 conceptually illustrates the refinement of prediction candidates (e.g., merged candidates) by bilateral matching (BM). MV0 is an initial motion vector or prediction candidate and MV1 is a mirror image of MV0. The current block 800 in the current image 801 refers to the initial reference block 820 in the reference image 810 via MV0. MV1 refers to the initial reference block 821 in the reference image 811. The figure shows that MV0 and MV1 are refined to form MV0' and MV1', which update reference blocks 830 and 831, respectively. The refinement is performed based on bilateral matching so that the refined motion vector pair MV0' and MV1' has a better bilateral matching cost than the initial motion vector pair MV0 and MV1. MV0'-MV0 (i.e., MVD0) and MV1'-MV1 (i.e., MVD1) are constrained to be equal in magnitude but opposite in direction. In some embodiments, the bilateral matching cost of a pair of mirrored motion vectors (e.g., MV0 and MV1) is calculated based on the difference between two reference blocks referenced by the mirrored motion vectors (e.g., the difference between reference blocks 820 and 821).

g. In-frame prediction

The intra prediction method uses a reference tier and an intra prediction mode adjacent to the current prediction unit (PU) to generate a predictor for the current PU. The intra prediction direction can be selected from a mode set containing multiple prediction directions. For each PU encoded and decoded by intra prediction, an index will be used and encoded to select an intra prediction mode. The corresponding prediction will be generated, and then the residual can be derived and transformed.

Figure 9 shows the intra prediction modes of different directions. These intra prediction modes are called directional modes and do not include DC mode or planar mode. As shown in the figure, there are 33 directional modes (V: vertical direction; H: horizontal direction), so H, H+1~H+8, H-1~H-7, V, V+1~V+8, V-1~V-8 are used. Usually the directional mode can be expressed as H+k or V+k mode, where k=±1, ±2,..., ±8. Each of these intra prediction modes can also be called an intra prediction angle. In order to capture arbitrary edge directions presented in natural video, the number of directional intra modes can be expanded from 33 used in HEVC to 65 directional modes so that k ranges from ±1 to ±16. These denser directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction. By including DC and planar modes, the number of intra prediction modes is 35 (or 67).

Among the 35 (or 67) intra-frame prediction modes, some modes (e.g., 3 or 5) are identified as a set of most probable modes (MPMs) for intra-frame prediction in the current prediction block. The encoder can reduce the bit rate by signaling the index of selecting one of the MPMs instead of selecting the index of one of the 35 (or 67) intra-frame prediction modes. For example, the intra-frame prediction mode used in the left prediction block and the intra-frame prediction mode used in the prediction block above are used as MPMs. When the intra-frame prediction modes in two neighboring blocks use the same intra-frame prediction mode, the intra-frame prediction mode can be used as the MPM. When only one of the two neighboring blocks is available and encoded and decoded in a directional mode, two neighboring directions adjacent to the directional mode can be used as MPMs. DC mode and planar mode are also considered as MPMs to fill available spots in the MPM set, in particular if the above or top neighboring block is not available or is not encoded in intra prediction, or if the intra prediction mode in the neighboring block is not a directional mode. If the intra prediction mode of the current prediction block is one of the modes in the MPM set, 1 or 2 bits are used to signal which one it is. Otherwise, the intra prediction mode of the current block is different from any entry in the MPM set, and the current block will be encoded as a non-MPM mode. There are a total of 32 such non-MPM modes, and a (5-bit) fixed-length codec method should be used to signal such modes.

The MPM list is constructed based on the in-frame modes of the left neighbor block and the upper neighbor block. Assuming that the mode of the left neighbor block is denoted as Left and the mode of the upper neighbor block is denoted as Above, the unified MPM list can be constructed as follows: – When a neighbor block is unavailable, its in-frame mode is set to flat by default. – If both Left and Above modes are non-angular modes: § 　 MPM list à{Planar, DC, V, H, V − 4, V + 4} – If one of Left and Above modes is angular and the other is non-angular: § 　 Set mode Max to the larger of Left and Above § 　 MPM list à{Planar, Max, Max − 1, Max + 1, Max − 2, Max + 2} – If both Left and Above are angular and they are different: § Set mode Max to the larger of Left and Above § Set mode Min to the smaller of Left and Above § If Max – Min is equal to 1: – MPM list à{Planar, Left, Above, Min – 1, Max + 1, Min – 2} § Otherwise, if Max – Min Greater than or equal to 62: – MPM list à{Planar, Left, Above, Min + 1, Max – 1, Min + 2} § Otherwise, if Max – Min is equal to 2: – MPM list à{Planar, Left, Above, Min + 1, Min – 1, Max + 1} § Otherwise: – MPM list à{Planar, Left, Above, Min – 1, Min + 1, Max – 1} § If both Left and Above are angled and they are the same: – 　 MPM list à{Planar, Left, Left − 1, Left + 1, Left – 2, Left + 2}

The traditional angular intra-frame prediction direction is defined as from 45 degrees to -135 degrees clockwise. In VVC, several traditional angular intra-frame prediction modes are adaptively replaced with non-square block wide-angle intra-frame prediction modes. The replaced modes are signaled using the original mode indexes, which are remapped to the indexes of the wide-angle mode after parsing.

For some embodiments, the total number of intra-frame prediction modes remains unchanged, i.e., 67, and the intra-frame mode encoding and decoding method remains unchanged. To support these prediction directions, a top reference template of length 2W+1 and a left reference template of length 2H+1 are defined. FIGS. 10A-B conceptually illustrate wide-angle directional modes with extended lengths of top and left reference templates to support non-square blocks of different aspect ratios.

h. Overlapping Block Motion Compensation (OverlappedBlockMotionCompensation , abbreviated as OBMC)

Overlapping Block Motion Compensation (OBMC) is a video codec tool that reduces blocking effects by refining the top and left boundary pixels of a CU using motion information from neighboring blocks with weighted prediction, specifically: Inter' _predY = ((128 – w ₁ ) × Inter _predY + w ₁ × OBMC _predY ) / 128 PredY = ((4 – w ₀ ) × FwdMap (Inter' _predY ) + w ₀ × Intra _predY ) / 4

Among them, Inter _predY represents the sample of motion prediction of the current block in the original domain, Intra _predY represents the sample predicted in the mapping domain, OBMC _predY represents the sample of motion prediction of the neighboring blocks in the original domain, and _w0 and _w1 are weighted.

Subblock boundary OBMC is performed by applying the same blending to the top, left, bottom, and right subblock boundary pixels using the motion information of neighboring subblocks. OBMC can be enabled for subblock-based coding tools such as Affine AMVP mode, Affine Merging mode, and Subblock-based Temporal Motion Vector Prediction, as well as Subblock-based Bilateral Matching. When the OBMC mode uses Luma Mapping with Chroma Scaling (LMCS) in CIIP mode, inter-frame blending is performed before LMCS mapping of inter-frame samples. LMCS is applied to the blended inter-frame samples, which are combined with intra-frame samples to which LMCS is applied in CIIP mode. III. Out-Of-Boundary Check for Prediction Coding Tools

In order to improve the coding efficiency of bidirectional motion compensation with OOB check, some embodiments of the present disclosure provide a method for performing OOB check for predictive coding modes other than bidirectional motion compensation.

A. ERP Video OOB Check

In some embodiments, for the horizontal wrap around motion compensation process or for motion compensation of 360-degree video in equirectangular projection (ERP) format, bidirectional motion compensation with OOB check is applied only in the vertical direction, i.e., horizontal wrap around motion compensation is applied to OOB samples in the horizontal direction, and bidirectional motion compensation with OOB check is applied to OOB samples in the vertical direction. In some embodiments, for the horizontal wrap around motion compensation process or for motion compensation of 360-degree video in ERP projection format, bidirectional motion compensation with OOB check is not applied.

FIG. 11A-C conceptually illustrates horizontal surround motion compensation with out-of-bounds (OOB) checking. The current block 1105 in the current picture 1100 is encoded and decoded by bidirectional prediction using motion vectors MV0 and MV1. FIG. 11A shows MV0 pointing to reference block 1120 in reference picture 1110. FIG. 11B shows MV1 pointing to reference block 1121 in L1 reference picture 1111.

As shown, a portion 1130 of the reference block 1120 is OOB because it is outside the left (or right) boundary of the projection domain of the reference image 1110. Instead of repeat padding, samples for the OOB portion 1130 of the reference block 1120 are generated by horizontal orbital motion compensation, specifically by copying from a spherical neighbor 1131 located within the reference image 1110 to the right boundary in the projection domain. A portion 1140 of the reference block 1121 is OOB because it is outside the top (or bottom) boundary of the projection domain of the L1 reference image 1111. In some embodiments, repeat padding is applied to generate samples for the OOB portion 1140.

FIG. 11C illustrates the generation of a bidirectional motion compensated predictor 1135 based on OOB and non-OOB samples of ERP video. For some embodiments, the method described in Section I above may be used to generate a predictor 1135 for the current block 1110 based on reference blocks 1120 and 1121 and their respective OOB states. For some embodiments, samples generated by the horizontal wrapping of the OOB portion 1130 of the reference block 1120 are not considered OOB samples.

As shown, at positions not corresponding to OOB samples, non-OOB samples from reference block 1120 (including horizontal surround samples 1130) and non-OOB samples from reference block 1121 are mixed (by, for example, weighted averaging) to generate prediction samples for predictor 1135. On the other hand, at sample positions corresponding to OOB samples (e.g., positions in OOB portion 1140), non-OOB samples from reference block 1120 (including horizontal surround samples 1130) are used (thus, OOB samples are not included in predictor 1135).

B. Prediction block within the frame OOB Check

In some embodiments, OOB check is applied to blocks predicted intra-frame. In some embodiments, if a sample is generated by a padded L-neighbor in an intra-frame block (e.g., at a picture boundary), the video codec may treat the sample as OOB. FIG. 12 conceptually illustrates intra-frame prediction using OOB reference samples. As shown, the current block 1210 is intra-frame predicted. Samples of the upper/top L-neighbor 1220 of the current block are true reconstructed samples, while samples of the left L-neighbor 1222 of the current block are padded samples. Block 1210 is intra-frame predicted from a diagonal direction. The samples in the upper triangle 1230 are non-OOB and the samples in the lower triangle 1232 are OOB samples.

C. For use with decoder side MV Thinned blocks OOB

In some embodiments, OOB checking is applied to DMVR sub-blocks (or codec-block-level MP-DMVR, or sub-block-level MP-DMVR). Specifically, if a search point (e.g., initial MV and MV offset) in the L0 and/or L1 reference pictures is OOB, the search point is designated as unavailable or deprioritized relative to non-OOB search points (e.g., by applying an additional cost or a predefined large cost). In some embodiments, if all search points and the initial MV are OOB, the refined MV is set to the initial MV.

In some embodiments, even if the initial MV is OOB, the video codec still compares the initial MV with other non-OOB search points. The video codec does not set the initial MV as unavailable, or set the initial MV to a lower priority than non-OOB search points, or add additional costs, or apply a predefined large cost.

D. The sub-blocks in the reference image OOB

In some embodiments, when bidirectional prediction uses a subblock in L0 and/or L1 reference pictures, a subblock in L0 or L1 reference pictures is OOB if samples in the subblock are OOB. In some embodiments, a subblock is OOB if the number of OOB samples in the subblock is greater than a predefined threshold, or if the ratio of OOB samples in the subblock is greater than a predefined ratio, or if all samples in the subblock are OOB. In some embodiments, a search point is OOB if both subblocks in L0 and L1 reference pictures are OOB, or any one of the subblocks in L0 or L1 reference pictures is OOB.

E.   GPM Coded Block OOB

In some embodiments, OOB check is applied to two (or more) partitions of a GPM codec block (via, for example, GPM-TM, GPM-MMVD, GPM-INTER-INTRA, GPM-INTRA-INTRA). Specifically, OOB check may be performed on 2 inter-frame partitions, or 1 inter-frame partition and 1 intra-frame partition, or 2 intra-frame partitions. GPM blending is then applied to the result of the OOB check according to the following: If P _i,j ^part0 is OOB and P _i,j ^part1 is non-OOB, then P _i,j ^final = P _i,j ^part1 Otherwise If P _i,j ^part0 is non-OOB and P _i,j ^part1 is OOB, then P _i,j ^final = P _i,j ^part0 Otherwise P _i,j ^final = GPM_blending (P _i,j ^part0 , P _i,j ^part1 )

In some embodiments, OOB check is not applied to intra-frame parts but only to inter-frame parts, so that the OOB check result of intra-frame parts is always non-OOB. The GPM blending of a GPM with one inter-frame part and one intra-frame part can be calculated as follows: If Pi _,j ^partX is OOB and partX is inter-frame and part 1-X is intra-frame, then Pi _,j ^final = Pi _,j ^part(1-X) otherwise Pi _,j ^final = GPM_blending (Pi _,j ^part0 , Pi _,j ^part1 )

If the samples of the inter-frame part are OOB, the samples of the intra-frame part are used as predictors. Otherwise, mixing is applied to the samples of the two parts. The GPM mixing of the GPM with two intra-frame parts is the same as the GPM mixing of the GPM with two inter-frame parts. In some embodiments, if the inter-frame prediction signal of the inter-frame part is bidirectional prediction, the OOB check result can be obtained from the OOB result of L0 reference and the OOB result of L1 reference: If the inter-frame part is bidirectional prediction, P _i,j ^Inter 's OOB = P _i, j ^{Inter,OOB of L0} & P _i,j ^{Inter,OOB of L1} Otherwise if the inter-frame part is bidirectional prediction (alternative method): P _i,j ^Inter 's OOB = P _i,j ^{Inter,OOB of L0} | P _i,j ^{Inter,OOB of L1}

F.   CIIP Codec Block OOB

In some embodiments, OOB check is applied to the inter-frame part and the intra-frame part of CIIP, and then CIIP mixing is applied according to the following results of the OOB check: If the inter-frame part is unidirectional prediction (L0): OOB of P _i,j ^Inter = OOB of P _i, j ^Inter,L0 If the inter-frame part is unidirectional prediction (L1): OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L1 If the inter-frame part is bidirectional prediction: OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L0 & OOB of P _i,j ^Inter,L1 Or if the inter-frame part is bidirectional prediction (alternative method): OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L0 | OOB of P _i,j ^Inter,L1

Then: if P _i,j ^Inter is OOB and P _i,j ^Intra is non-OOB P _i,j ^final = P _i,j ^Intra else if P _i,j ^Inter is non-OOB and P _i,j ^Intra is OOB P _i,j ^final = P _i,j ^Inter else P _i,j ^final = CIIP_blending (P _i,j ^Inter , P _i,j ^Intra )

In some embodiments, OOB check is not applied to the intra-frame part but only to the inter-frame part, i.e., the OOB check result of the intra-frame part is always non-OOB. Then CIIP blending of a CIIP with one inter-frame part and one intra-frame part is performed according to the following: If Pi _,j ^Inter is OOB Pi _,j ^final = Pi _,j ^Intra Otherwise Pi _,j ^final = CIIP_blending (Pi _,j ^Inter , Pi _,j ^Intra )

If the sample in the inter-frame part is OOB, the corresponding sample in the intra-frame part is used as the predictor, otherwise the CIIP blending is applied to the samples in the two CIIP parts.

G. For template matching ( TemplateMatching )of OOB

In some embodiments, OOB check is applied to the template region of template matching (TM) or ARMC-TM, and then the application of the results of the OOB check is mixed. For example, the video codec can perform motion vector refinement based on minimizing the TM matching cost between a reference template (e.g., reference template 630) adjacent to the reference block and a current template (e.g., current template 620) adjacent to the current block. The TM cost can be calculated based on the OOB check of the reference template and/or the current template. For example, OOB samples in the reference template will not be used to determine the TM cost.

H.   OBMC of OOB

In some embodiments, OOB check is applied to overlapped block motion compensation (OBMC), and then the additional inter-frame prediction is mixed with the result of the OOB check. If the OBMC inter-frame prediction signal is OOB, the OBMC inter-frame prediction signal is not mixed with the original prediction signal. Otherwise, if the OBMC inter-frame prediction signal is not OOB, the OBMC inter-frame prediction signal is mixed with the original prediction signal. In some embodiments, if the additional inter-frame prediction signal is a bidirectional prediction, the OOB check result can be derived from the OOB result of the L0 prediction (or reference block) and the OOB result of the L1 prediction (or reference block) according to the following: If the inter-frame part is bidirectional prediction: OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L0 & OOB of P _i,j ^Inter,L1 Otherwise if the inter-frame part is bidirectional prediction (alternative method): OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L0 | OOB of P _i,j ^Inter,L1

I .    MHP of OOB

In some embodiments, the video codec applies an OOB check to a multiple hypothesis prediction (MHP) and then adds the additional inter-frame prediction signal/hypothesis to the result of the OOB check. If the additional inter-frame prediction signal is OOB, the additional inter-frame prediction signal is not added to the original prediction signal. Otherwise, if the additional inter-frame prediction signal is not OOB, the additional inter-frame prediction signal is added to the original prediction signal. In some embodiments, if the additional inter-frame prediction signal is a bidirectional prediction, the OOB check result can be derived from the OOB result of the L0 prediction (reference block) and the OOB result of the L1 prediction (reference block) as follows: If the inter-frame part is bidirectional prediction: OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L0 & OOB of P _i,j ^Inter,L1 Otherwise if the inter-frame part is bidirectional prediction (alternative method): OOB of P _i,j ^Inter = OOB of P _i,j ^Inter,L0 | OOB of P _i,j ^Inter,L1

The aforementioned proposed method can be implemented in an encoder and/or a decoder. For example, the proposed method can be implemented in an inter-frame prediction module and/or an intra-frame block copy prediction module of an encoder and/or an inter-frame prediction module (and/or an intra-frame block copy prediction module) of a decoder. IV . Example Video Encoder

FIG. 13 shows an example video encoder 1300 that may implement predictive coding. As shown, the video encoder 1300 receives an input video signal from a video source 1305 and encodes the signal into a bit stream 1395. The video encoder 1300 has several components or modules for encoding a signal from a video source 1305, including at least some components selected from the following: a transform module 1310, a quantization module 1311, an inverse quantization module 1314, an inverse transform module 1315, an intra-picture estimation module 1320, an intra-frame prediction module 1325, a motion compensation module 1330, a motion estimation module 1335, a loop filter 1345, a reconstructed picture buffer 1350, an MV buffer 1365, an MV prediction module 1375, and an entropy encoder 1390. The motion compensation module 1330 and the motion estimation module 1335 are part of the inter-frame prediction module 1340.

In some embodiments, modules 1310-1390 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 1310-1390 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 1310-1390 are shown as separate modules, some modules may be combined into a single module.

The video source 1305 provides a raw video signal that presents pixel data for each video frame without compression. The subtractor 1308 calculates the difference between the raw video pixel data of the video source 1305 and the predicted pixel data 1313 from the motion compensation module 1330 or the intra-frame prediction module 1325. The transform module 1310 transforms the difference (or residual pixel data or residual signal 1308) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 1311 quantizes the transform coefficients into quantized data (or quantized coefficients) 1312, which are encoded into a bit stream 1395 by the entropy encoder 1390.

The inverse quantization module 1314 dequantizes the quantized data (or quantized coefficients) 1312 to obtain transform coefficients, and the inverse transform module 1315 performs inverse transform on the transform coefficients to generate reconstruction residues 1319. The reconstruction residues 1319 are added to the predicted pixel data 1313 to generate reconstructed pixel data 1317. In some embodiments, the reconstructed pixel data 1317 is temporarily stored in a line buffer (linebuffer not shown) for intra-picture prediction and spatial MV prediction. The reconstructed pixels are filtered by a loop filter 1345 and stored in a reconstructed picture buffer 1350. In some embodiments, the reconstructed picture buffer 1350 is a memory outside the video encoder 1300. In some embodiments, the reconstructed picture buffer 1350 is a memory inside the video encoder 1300.

The intra-picture estimation module 1320 performs intra-frame prediction based on the reconstructed pixel data 1317 to generate intra-frame prediction data. The intra-frame prediction data is provided to the entropy encoder 1390 to be encoded into a bit stream 1395. The intra-frame prediction data is also used by the intra-frame prediction module 1325 to generate the predicted pixel data 1313.

The motion estimation module 1335 performs inter-frame prediction by generating MVs to refer to pixel data of previously decoded frames stored in the reconstructed picture buffer 1350. These MVs are provided to the motion compensation module 1330 to generate predicted pixel data.

Instead of encoding the complete actual MV in the bitstream, the video encoder 1300 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 1395.

Based on the reference MV generated for encoding the previous video frame, i.e., the motion compensation MV for performing motion compensation, the MV prediction module 1375 generates a predicted MV. The MV prediction module 1375 obtains the reference MV from the previous video frame from the MV buffer 1365. The video encoder 1300 stores the MV generated for the current video frame in the MV buffer 1365 as a reference MV for generating the predicted MV.

The MV prediction module 1375 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the predicted MV and the motion compensation MV (MCMV) of the current frame is encoded by the entropy encoder 1390 into the bit stream 1395.

The entropy encoder 1390 encodes various parameters and data into the bit stream 1395 by using entropy coding and decoding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 1390 encodes various header elements, flags along with quantized transform coefficients 1312 and residual motion data as syntax elements into the bit stream 1395. The bit stream 1395 is then stored in a storage device or transmitted to a decoder via a communication medium such as a network.

The loop filter 1345 performs filtering or smoothing operations on the reconstructed pixel data 1317 to reduce encoding and decoding artifacts, especially at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operations performed by the loop filter 1345 include deblocking filter (DBF), sample adaptive offset (SAO) and/or adaptive loop filter (ALF).

FIG. 14 illustrates a portion of a video encoder 1300 that implements out-of-bounds checking for predictive coding. Specifically, the figure shows the components of a motion compensation module 1330 of the video encoder 1300.

As shown, the motion compensation module 1330 has a candidate selector 1410, which selects a block vector (including a motion vector and an intra-frame prediction mode) from the MV buffer 1365 and the intra-frame prediction module 1325. The selection is controlled by the motion estimation module 1335. The selected block vector is checked by the OOB check module 1430, which determines whether the reference block identified by the block vector is out of bounds. The selected block vector is also provided to the predictor generator module 1420.

The predictor generator module 1420 generates a predictor or prediction block for the current block by obtaining samples from the reconstructed picture buffer 1350 as the predicted pixel data 1313. The motion estimation module 1335 selects one or more prediction codec tools (e.g., bidirectional prediction, GPM, CIIP, MHP, intra-frame prediction, IBC, etc.) for the current block, and the predictor generator module 1420 generates a predictor based on the selected prediction tool. The motion estimation module 1335 also provides the prediction tool selection to the entropy encoder 1390 for signaling in the bitstream 1395.

The predictor generator module 1420 also uses the OOB check result from the OOB check module 1430 to determine whether to exclude certain reference samples identified by the selected block vector when performing blending to generate the predictor. For example, if the current picture is an ERP picture and the reference block identified by the block vector has OOB samples, the predictor generator module 1420 can use the horizontal surround samples obtained from the reconstructed picture buffer 1650 for blending if the reference block is OOB at the left or right boundary, or exclude the OOB samples from blending if the reference block is OOB at the top or bottom boundary. The predictor generator module 1420 can also use the OOB result to determine whether to exclude the sample from the BM cost calculation or the TM cost calculation.

FIG. 15 conceptually illustrates a process 1500 for predictive encoding of a pixel block using out-of-bounds checking. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the encoder 1300 perform the process 1500 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the encoder 1300 performs the process 1500.

The encoder receives (at block 1510) data for a current pixel block in a current picture to be encoded as a video. The encoder identifies (at block 1520) a first reference block in a first reference picture based on a first block vector of the current block. The encoder identifies (at block 1530) a second reference block in a second reference picture based on a second block vector of the current block. The first and second block vectors may be motion vectors. In some embodiments, the first reference picture or the second reference picture is the current picture, and the first block vector or the second block vector is an intra-frame prediction direction or mode.

The encoder (at block 1540) performs an out-of-bounds (OOB) check on the first and second reference blocks. The encoder (at block 1550) generates a predictor for the current block based on the first and second reference blocks and based on the OOB check. In some embodiments, when the sample of the first reference block is OOB, bidirectional motion compensation is not applied for at least one of the horizontal direction and the vertical direction. In some embodiments, when the sample of the first reference block is OOB on the left or right boundary of the first reference picture, or when the sample of the first reference block is OOB on the top boundary or the bottom boundary of the first reference picture, bidirectional motion compensation is not applied. In some embodiments, when the samples of the first reference block are non-OOB samples, the corresponding prediction samples are generated by mixing the non-OOB samples of the first reference block with the corresponding non-OOB samples of the second reference block. When the samples of the first reference block are OOB samples, the corresponding prediction samples are generated using the corresponding non-OOB samples of the second reference block without mixing.

In some embodiments, the video may be a 360-degree video in an equirectangular projection (ERP) format. When samples of a first reference block are OOB on a left or right border of a first reference picture, a corresponding prediction sub-sample is generated by blending horizontal surround samples from the first reference picture with corresponding non-OOB samples from a second reference block in a second reference picture. When samples of the first reference block are OOB on a top or bottom border of the first reference picture, a corresponding prediction sub-sample is generated by using corresponding non-OOB samples from a second reference block without blending.

In some embodiments, when the current block is divided into a plurality of geometric partitions, a first reference block is used to generate a first predictor for the first geometric partition and a second reference block is used to generate a second predictor for the second geometric partition. The corresponding predictor samples are generated by blending samples of the first and second predictors along a boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predictors are excluded from the blend.

In some embodiments, the generated predictor is a combined inter-frame intra-frame prediction (CIIP), wherein a first reference block provides inter-frame prediction samples and a second reference block provides intra-frame prediction samples. In some embodiments, the current block is encoded using MHP, so that the encoder performs an additional OOB check on a third reference block, and further generates a predictor based on the third reference block and the additional OOB check.

In some embodiments, the first block vector is improved to minimize a calculated template matching cost between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein OOB samples of the reference template are excluded from the cost calculation. In some embodiments, the first and second block vectors are improved to minimize a bilateral matching cost based on the first and second reference blocks, wherein OOB samples in the first and second reference blocks are excluded from the cost calculation.

The encoder (at block 1560) encodes the current block using the generated predictor to produce a prediction residual. V. Example Video Decoder

In some embodiments, an encoder may send (or generate) one or more syntax elements in a bitstream so that a decoder may parse the one or more syntax elements from the bitstream.

FIG. 16 shows an example video decoder 1600 that can implement predictive coding. As shown, the video decoder 1600 is an image decoding or video decoding circuit that receives a bit stream 1695 and decodes the content of the bit stream into pixel data of a video frame for display. The video decoder 1600 has several components or modules for decoding the bit stream 1695, including components selected from the following: an inverse quantization module 1611, an inverse transform module 1610, an intra-frame prediction module 1625, a motion compensation module 1630, a loop filter 1645, a decoded picture buffer 1650, an MV buffer 1665, an MV prediction module 1675, and a parser 1690. The motion compensation module 1630 is part of the frame prediction module 1640.

In some embodiments, modules 1610-1690 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1610-1690 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1610-1690 are shown as separate modules, some modules may be combined into a single module.

The parser 1690 (or entropy decoder) receives the bitstream 1695 and performs initial parsing according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1612. The parser 1690 uses entropy coding and decoding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman encoding.

The inverse quantization module 1611 dequantizes the quantized data (or quantized coefficients) 1612 to obtain transform coefficients, and the inverse transform module 1610 performs inverse transform on the transform coefficients 1616 to generate a reconstructed residual signal 1619. The reconstructed residual signal 1619 is added with the predicted pixel data 1613 from the intra-frame prediction module 1625 or the motion compensation module 1630 to generate decoded pixel data 1617. The decoded pixel data is filtered by the loop filter 1645 and stored in the decoded picture buffer 1650. In some embodiments, the decoded picture buffer 1650 is a memory outside the video decoder 1600. In some embodiments, the decoded picture buffer 1650 is a memory inside the video decoder 1600.

The intra prediction module 1625 receives intra prediction data from the bitstream 1695 and, based thereon, generates predicted pixel data 1613 from decoded pixel data 1617 stored in the decoded picture buffer 1650. In some embodiments, the decoded pixel data 1617 is also stored in a line buffer (not shown) for intra picture prediction and spatial MV prediction.

In some embodiments, the contents of the decoded picture buffer 1650 are used for display. The display device 1655 either obtains the contents of the decoded picture buffer 1650 for direct display or obtains the contents of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values from the decoded picture buffer 1650 by pixel transfer.

The motion compensation module 1630 generates predicted pixel data 1613 from the decoded pixel data 1617 stored in the decoded picture buffer 1650 according to the motion compensation MV (MC MV). These motion compensation MVs are decoded by adding the residual motion data received from the bitstream 1695 to the predicted MV received from the MV prediction module 1675.

Based on a reference MV generated for decoding a previous video frame (e.g., a motion compensation MV for performing motion compensation), the MV prediction module 1675 generates a predicted MV. The MV prediction module 1675 obtains the reference MV of the previous video frame from the MV buffer 1665. The video decoder 1600 stores the motion compensation MV generated for decoding the current video frame in the MV buffer 1665 as a reference MV for generating a predicted MV.

The loop filter 1645 performs a filtering or smoothing operation on the decoded pixel data 1617 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering or smoothing operation performed by the loop filter 1645 includes deblocking filtering (DBF), sample adaptive offset (SAO), and/or adaptive filtering (ALF).

FIG. 17 illustrates a portion of a video decoder 1600 that implements out-of-bounds checking for predictive coding. Specifically, the figure shows the components of a motion compensation module 1630 of the video decoder 1600.

As shown, the motion compensation module 1630 has a candidate selector 1710, which selects a block vector (including motion vectors and intra-frame prediction mode) from the MV buffer 1665 and the intra-frame prediction module 1625. The selection is provided by the entropy decoder 1690 based on the syntax elements parsed from the bit stream 1695. The selected block vector is checked by the OOB check module 1730, which determines whether the reference block identified by the block vector is out of bounds. The selected block vector is also provided to the predictor generator module 1720.

The predictor generator module 1720 generates a predictor or a prediction block of the current block by obtaining samples from the decoded picture buffer 1650 as predicted pixel data 1613. The entropy decoder 1690 selects one or more prediction codec tools (e.g., bidirectional prediction, GPM, CIIP, MHP, intra-frame prediction, IBC, etc.) for the current block, and the predictor generator module 1720 generates a predictor based on the selected prediction tool.

The predictor generator module 1720 also uses the OOB check result from the OOB check module 1730 to determine whether to exclude certain reference samples identified by the selected block vector when performing blending to generate the predictor. For example, if the current picture is an ERP picture and the reference block identified by the block vector has OOB samples, the predictor generator module 1720 can use the horizontal surround samples obtained from the decoded picture buffer 1650 for blending if the reference block is OOB at the left or right boundary, or exclude the OOB samples from blending if the reference block is OOB at the top or bottom boundary. The predictor generator module 1720 can also use the OOB result to determine whether to exclude samples from the BM cost calculation or the TM cost calculation.

FIG. 18 conceptually illustrates a process 1800 for predictive encoding and decoding of a pixel block using out-of-bounds checking. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing the decoder 1600 perform the process 1800 by executing instructions stored in a computer-readable medium. In some embodiments, an electronic device implementing the decoder 1600 performs the process 1800.

The decoder receives (at block 1810) data of a current pixel block in a current picture to be decoded as a video. The decoder identifies (at block 1820) a first reference block in a first reference picture based on a first block vector of the current block. The decoder identifies (at block 1830) a second reference block in a second reference picture based on a second block vector of the current block. The first and second block vectors may be motion vectors. In some embodiments, the first reference picture or the second reference picture is the current picture, and the first block vector or the second block vector is an intra-frame prediction direction or mode.

The decoder (at block 1840) performs an out-of-bounds (OOB) check on the first and second reference blocks. The decoder (at block 1850) generates a predictor for the current block based on the first and second reference blocks and based on the OOB check. In some embodiments, when the sample of the first reference block is OOB, bidirectional motion compensation is not applied for at least one of the horizontal direction and the vertical direction. In some embodiments, when the sample of the first reference block is OOB on the left or right boundary of the first reference picture, or when the sample of the first reference block is OOB on the top boundary or the bottom boundary of the first reference picture, bidirectional motion compensation is not applied. In some embodiments, when the samples of the first reference block are non-OOB samples, the corresponding prediction samples are generated by mixing the non-OOB samples of the first reference block with the corresponding non-OOB samples of the second reference block. When the samples of the first reference block are OOB samples, the corresponding prediction samples are generated using the corresponding non-OOB samples of the second reference block without mixing.

In some embodiments, the video may be a 360-degree video in an equirectangular projection (ERP) format. When a sample of a first reference block is OOB on a left or right border of a first reference picture, a corresponding prediction sample is generated by blending horizontal surround samples from the first reference picture with corresponding non-OOB samples from a second reference block in a second reference picture. When a sample of the first reference block is OOB on a top or bottom border of the first reference picture, a corresponding prediction sub-sample is generated by using corresponding non-OOB samples from a second reference block without blending.

In some embodiments, when the current block is divided into a plurality of geometric partitions, a first reference block is used to generate a first predictor for the first geometric partition and a second reference block is used to generate a second predictor for the second geometric partition. The corresponding predictor samples are generated by blending samples of the first and second predictors along a boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predictors are excluded from the blend.

In some embodiments, the generated predictor is a combined inter-frame intra-frame prediction (CIIP), where a first reference block provides inter-frame prediction samples and a second reference block provides intra-frame prediction samples. In some embodiments, the current block is encoded and decoded using MHP, so that the decoder performs an additional OOB check on a third reference block, and further generates a predictor based on the third reference block and the additional OOB check.

In some embodiments, the first block vector is refined to minimize a template matching cost calculated between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein OOB samples of the reference template are excluded from the cost calculation. In some embodiments, the first and second block vectors are refined to minimize a bilateral matching cost based on the first and second reference blocks, wherein OOB samples in the first and second reference blocks are excluded from the cost calculation.

The decoder (at block 1860) reconstructs the current block by using the generated predictors. The decoder may then provide the reconstructed current block for display as part of a reconstructed current picture. VI. Example Electronic System

Many of the above features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, compact disc read-only memory (CD-ROM), flash memory drives, random-access memory (RAM) chips, hard disk drives, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media does not include carrier waves and electronic signals transmitted over wireless or wired connections.

In this specification, the term "software" is intended to include firmware residing in read-only memory or applications stored in magnetic memory that can be read into memory for processing by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement the software inventions described herein are within the scope of this disclosure. In some embodiments, the software program, when installed to run on one or more electronic systems, defines one or more specific machine implementations that process and execute the operations of the software program.

FIG. 19 conceptually illustrates an electronic system 1900 for implementing some embodiments of the present disclosure. The electronic system 1900 may be a computer (e.g., a desktop computer, a personal computer, a tablet computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1900 includes a bus 1905, a processing unit 1910, a graphics processing unit (GPU) 1915, a system memory 1920, a network 1925, a read-only memory 1930, a permanent storage device 1935, an input device 1940, and an output device 1945.

Buses 1905 collectively represent all system, peripheral, and chipset buses that communicatively couple the numerous internal devices of electronic system 1900. For example, bus 1905 communicatively couples processing unit 1910 with GPU 1915, read-only memory 1930, system memory 1920, and permanent storage 1935.

The processing unit 1910 obtains instructions to be executed and data to be processed from these various memory units in order to perform the processing disclosed herein. In different embodiments, the processing unit can be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 1915. The GPU 1915 can offload various calculations or supplement the image processing provided by the processing unit 1910.

Read-only-memory (ROM) 1930 stores static data and instructions used by processing unit 1910 and other modules of the electronic system. On the other hand, permanent storage device 1935 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when the electronic system 1900 is turned off. Some embodiments of the present disclosure use a large-capacity memory device (such as a disk or optical disk and its corresponding disk drive) as permanent storage device 1935.

Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1935, system memory 1920 is a read-write memory device. However, unlike permanent storage device 1935, system memory 1920 is a volatile read-write memory, such as random access memory. System memory 1920 stores some instructions and data used by the processor during operation. In some embodiments, processing according to the present disclosure is stored in system memory 1920, permanent storage device 1935 and/or read-only memory 1930. For example, according to some embodiments of the present disclosure, various memory units include instructions for processing multimedia clips. From these various memory units, processing unit 1910 obtains instructions to be executed and data to be processed in order to perform the processing of some embodiments.

Bus 1905 is also connected to input devices 1940 and output devices 1945. Input devices 1940 enable a user to communicate information and select commands to an electronic system. Input devices 1940 include alphanumeric keyboards and pointing devices (also known as "cursor control devices"), cameras (e.g., webcams), microphones, or similar devices for receiving voice commands, etc. Output devices 1945 display images or output data generated by the electronic system. Output devices 1945 include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD), as well as speakers or similar audio output devices. Some embodiments include devices that function as both input and output devices, such as a touch screen.

Finally, as shown in FIG. 19 , bus 1905 also couples electronic system 1900 to a network 1925 via a network interface card (not shown). In this manner, the computer can be part of a computer network, such as a local area network (“LAN”), a wide area network (“WAN”), or an intranet, or a network of multiple networks, such as the Internet. Any or all of the components of electronic system 1900 may be used in conjunction with the present disclosure.

Some embodiments include electronic components, such as microprocessors, storage devices, and memory, which store computer program instructions in machine-readable or computer-readable media (or referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, double-layer DVD-ROM), various recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD Card, Mini SD Card, Micro SD Card, etc.), magnetic and/or solid state hard disk drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer codes include machine codes such as those generated by a compiler, and documents including high-level codes executed by a computer, an electronic component, or a microprocessor using an interpreter.

Although the above discussion primarily involves microprocessors or multi-core processors executing software, many of the above features and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

As used in this specification and any claims hereof, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical equipment. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display refer to displaying on an electronic device. As used in this specification and any claims hereof, the terms "computer-readable medium," "computer-readable medium," and "machine-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other transient signals.

Although the present disclosure has been described with reference to many specific details, a person of ordinary skill in the art will recognize that the present disclosure may be implemented in other specific forms without departing from the spirit of the present disclosure. In addition, many figures (including Figures 15 and 18) conceptually illustrate the processing. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous series of operations, and different specific operations may be performed in different embodiments. In addition, the processing may be implemented using several sub-processes, or as part of a larger macro-process. Therefore, a person of ordinary skill in the art will understand that the present disclosure is not limited by the foregoing illustrative details, but is limited by the scope of the attached patent application. Supplementary description

The subject matter described herein sometimes represents different components, which are contained in or connected to other different components. It is understood that the described structure is only an example, and can actually be implemented by many other structures to achieve the same function. Conceptually, any arrangement of components that achieve the same function is actually "associated" in order to achieve the desired function. Therefore, regardless of the structure or intermediate components, any two components combined to achieve a specific function are considered to be "interrelated" to achieve the desired function. Similarly, any two associated components are considered to be "operably connected" or "operably coupled" to each other to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve a specific function. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve a specific function. Specific examples of operable connections include, but are not limited to, physically mateable and/or physically interacting components, and/or wirelessly interactable and/or wirelessly interacting components, and/or logically interacting and/or logically interactable components.

In addition, with respect to the use of substantially any plural and/or singular terms, those of ordinary skill in the art can convert from the plural to the singular and/or from the singular to the plural according to context and/or application. For clarity, the present invention expressly describes different singular/plural arrangements.

In addition, it will be understood by those of ordinary skill in the art that, generally, the terms used in the present invention, especially in the claims, such as the subject matter of the claims, are generally used as "open" terms, for example, "including" should be interpreted as "including but not limited to", "having" should be interpreted as "at least having", "including" should be interpreted as "including but not limited to", etc. It will be further understood by those of ordinary skill in the art that if a specific number of claims are planned to be introduced, it will be clearly indicated in the claims, and it will not be displayed when there is no such content. For example, to help understanding, the claims below may contain the phrases "at least one" and "one or more" to introduce the claims. However, the use of these phrases should not be understood to imply the use of the indefinite article "one" or "a kind" to introduce the claims, thereby limiting any specific claims. Even when the same claim includes the introductory phrase "one or more" or "at least one", the indefinite article, such as "an" or "an", should be interpreted to mean at least one or more, as is the case with the use of the explicit description used to introduce the claim. In addition, even if an introductory statement explicitly refers to a specific number, one of ordinary skill in the art would recognize that such statement should be interpreted to mean the number referred to, e.g., "two references" without other modifications means at least two references, or two or more references. In addition, when using expressions similar to "at least one of A, B, and C", it is usually expressed in such a way that a person of ordinary skill in the art can understand the expression, for example, "a system includes at least one of A, B, and C" will include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and/or a system having A, B, and C, etc. A person of ordinary skill in the art will further understand that any separated words and/or phrases represented by two or more alternative terms, whether in the specification, in the scope of the patent application, or in the drawings, should be understood to include the possibility of one of these terms, one of them, or both of these terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

As can be seen from the foregoing, various embodiments have been described for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the invention. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and application are indicated by the scope of the patent application.

100, 110, 111, 200, 210, 211, 212, 600, 601, 801, 810, 811, 1100, 1110, 1111: Image 105, 700, 800, 1105: Current block 120, 121, 820, 821, 1120, 1121: Reference block 130: Prediction generation 135: Prediction block 140, 1130, 1140: OOB part 230, 231, 241, 242, 243, 244, 1135: Prediction sub 400: GPM candidate list 500, 605: CU 610, 611: MV 620, 630, 631: Template 711-717, 721-727: sub-template 1131: spherical neighbor 1220, 1222: L neighbor 1230: non-OOB 1232: OOB 1300: encoder 1305: video source 1308: subtractor 1310: transform module 1311: quantization module 1312: transform coefficient 1313: predicted pixel data 1314: inverse quantization module 1315: inverse transform module 1316: transform coefficient 1317: reconstructed pixel data 1319: reconstruction residual 1320: intra-image estimation module 1325: intra-frame prediction module 1330: motion compensation module 1335: Motion estimation module 1340: Inter-frame prediction module 1345: Loop filter 1350: Reconstructed image buffer 1365: MV buffer 1375: MV prediction module 1390: Entropy encoder 1395: Bitstream 1310: Candidate partition module 1315: MV candidate identification module 1320: Template identification module 1330: Cost calculator 1340: Candidate selection module 1410, 1710: Candidate selector 1420, 1720: Prediction sub-generator module 1430, 1730: OOB check module 1500, 1800: Process 1500 1510~1560, 1810~1860: Steps 1600: Video decoder 1610: Inverse transform module 1611: Inverse quantization module 1612: Quantization data 1613: Prediction pixel data 1616: Transformation coefficient 1617: Decoded pixel data 1619: Reconstructed residual signal 1625: Intra-frame prediction module 1630: Motion compensation module 1640: Inter-frame prediction module 1645: Loop filter 1650: Decoded picture buffer 1655: Display device 1665: MV buffer 1675: MV prediction module 1690: Entropy decoder 1695: Bitstream 1610: Candidate partition module 1615: MV candidate identification module 1620: Template identification module 1630: Cost calculator 1640: Candidate selection module 1900: Electronic system 1905: Bus 1910: Processing unit 1915: GPU 1920: System memory 1925: Network 1930: Read-only memory 1935: Persistent storage device 1940: Input device 1945: Output device

The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate the implementation of the present disclosure and are used together with the description to explain the principles of the present disclosure. It is worth noting that the accompanying drawings are not necessarily drawn to scale, because in actual implementations, specific components may be shown as being out of proportion to the size in order to clearly illustrate the concepts of the present disclosure. Figure 1 illustrates bidirectional prediction using an out-of-boundary (OOB) reference block. Figure 2 conceptually illustrates a multiple hypothesis prediction (MHP) for encoding and decoding the current block. Figure 3 shows the partitioning of a CU by a geometric partitioning mode (GPM). Figure 4 shows an exemplary unidirectional prediction candidate list for GPM partitioning and the selection of a unidirectional prediction MV for GPM. FIG. 5 illustrates an example partition edge blending process for a GPM of a CU. FIG. 6 conceptually illustrates performing template matching based on a search region around an initial motion vector (MV). FIG. 7 conceptually illustrates a current block 700 with sub-block motion. FIG. 8 conceptually illustrates refinement of prediction candidates (e.g., merge candidates) by bilateral matching (BM). FIG. 9 shows intra prediction modes for different directions. FIGS. 10A-B conceptually illustrate wide angle direction modes with extended length top and left reference templates to support non-square blocks of different aspect ratios. FIGS. 11A-C conceptually illustrate horizontal surround motion compensation with out-of-bounds (OOB) check. FIG. 12 conceptually illustrates intra-frame prediction using OOB reference samples. FIG. 13 illustrates an example video encoder that may implement predictive coding. FIG. 14 illustrates a portion of a video encoder that implements out-of-bounds checking for predictive coding. FIG. 15 conceptually illustrates a process for predictive coding for a pixel block using out-of-bounds checking. FIG. 16 illustrates an example video decoder that may implement predictive coding. FIG. 17 illustrates a portion of a video decoder that implements out-of-bounds checking for predictive coding. FIG. 18 conceptually illustrates a process for predictive coding for a pixel block using out-of-bounds checking. FIG. 19 conceptually illustrates an electronic system that implements some embodiments of the present disclosure.

1500: Process

1510~1560: Steps

Claims (13)

A video encoding and decoding method, comprising: receiving data of a current block of a current picture to be encoded or decoded as a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied to at least one of the horizontal direction and the vertical direction; and encoding or decoding the current block using the generated predictor. The video encoding and decoding method as described in claim 1, wherein: When the sample of the first reference block is out of bounds on the left or right boundary of the first reference picture, the corresponding prediction sample is generated by mixing the horizontal surround sample from the first reference picture with the corresponding non-out-of-bounds sample from the second reference block in the second reference picture, When the sample of the first reference block is out of bounds on the top or bottom boundary of the first reference picture, the corresponding prediction sample is generated by using the corresponding non-out-of-bounds sample from the second reference block without mixing. A video encoding and decoding method as described in claim 2, wherein the video is a 360-degree video in an equirectangular projection format. The video encoding and decoding method as described in claim 1, wherein: When the sample of the first reference block is out of bounds on the left boundary or the right boundary of the first reference picture, or when the sample of the first reference block is out of bounds on the top boundary or the bottom boundary of the first reference picture, the bidirectional motion compensation is not applied. The video encoding and decoding method as described in claim 1, wherein the first reference picture or the second reference picture is the current picture. A video encoding and decoding method as described in claim 5, wherein the first block vector or the second block vector is an intra-frame prediction direction. A video encoding and decoding method as described in claim 1, wherein: the current block includes a plurality of geometric partitions, the first reference block is used to generate a first predictor for the first geometric partition, the second reference block is used to generate a second predictor for the second geometric partition, wherein corresponding predictor samples are generated by mixing samples of the first and second predictors along a boundary between the first and second geometric partitions, wherein out-of-bounds samples of the first and second predictors are excluded from the mixing. A video coding and decoding method as described in claim 1, wherein the generated predictor is a combined inter-frame and intra-frame prediction, wherein the first reference block provides inter-frame prediction samples and the second reference block provides intra-frame prediction samples. The video encoding and decoding method as described in claim 1 also includes performing an additional out-of-bounds check on a third reference block, wherein the predictor is further generated based on the third reference block and the additional out-of-bounds check. A video encoding and decoding method as described in claim 1, wherein the first block vector is refined to minimize the template matching cost calculated between a reference template adjacent to the first reference block and a current template adjacent to the current block, wherein out-of-bounds samples of the reference template are excluded from the cost calculation. A video coding and decoding method as described in claim 1, wherein the first and second block vectors are refined based on the first and second reference blocks to minimize the bilateral matching cost, wherein out-of-bounds samples in the first and second reference blocks are excluded from the cost calculation. An electronic device, comprising: A video encoder circuit, configured to perform operations, including: Receiving data of a current block of a current picture to be encoded or decoded as a video; Identifying a first reference block in a first reference picture based on a first block vector of the current block; Identifying a second reference block in a second reference picture based on a second block vector of the current block; Performing an out-of-bounds check on the first and second reference blocks; Generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied to at least one of the horizontal direction and the vertical direction; and Encoding or decoding the current block using the generated predictor. A video decoding method, comprising: receiving data of a current block of a current picture to be decoded as a video; identifying a first reference block in a first reference picture based on a first block vector of the current block; identifying a second reference block in a second reference picture based on a second block vector of the current block; performing an out-of-bounds check on the first and second reference blocks; generating a predictor of the current block based on the first and second reference blocks and based on the out-of-bounds check, wherein when a sample of the first reference block is out-of-bounds, bidirectional motion compensation is not applied to at least one of the horizontal direction and the vertical direction; and reconstructing the current block using the generated predictor.