WO2016165122A1

WO2016165122A1 - Inter prediction offset

Info

Publication number: WO2016165122A1
Application number: PCT/CN2015/076865
Authority: WO
Inventors: Han HUANG
Original assignee: Mediatek Singapore Pte. Ltd.
Priority date: 2015-04-17
Filing date: 2015-04-17
Publication date: 2016-10-20

Abstract

Inter prediction offset is proposed. Besides the motion information, an inter prediction offset is also derived from the neighboring coded block to enhance the motion compensation prediction, or to predict the residual signal.

Description

INTER PREDICTION OFFSET

FIELD OF INVENTION

The invention relates generally to image and video processing. In particular, the presented invention relates to image and video coding.

BACKGROUND OF THE INVENTION

Motion compensation prediction (MCP) is a powerful tool in video coding. Instead of the original signal, the prediction residual is coded together with the side information. The side information usually includes: motion vector (s) and the reference index of the reference picture (s) . In some special coding mode, merge mode in HEVC [1] for example, the side information of current block is derived from already coded block (s) to reduce the overhead of signaling the side information. The prediction residual signal is usually coded by transform coding.

SUMMARY OF THE INVENTION

Methods of deriving inter prediction offset are proposed. An inter prediction offset is derived from already coded block (s) to improve the efficiency of motion compensation in current block.

Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

Fig. 1 is a diagram illustrating an example of neighboring coded block chosen for offset calculation.

Fig. 2 is a diagram illustrating an example of Example of using multiple neighboring coded blocks for offset calculation.

Fig. 3 is a diagram illustrating an example of spatial merge candidates in HEVC.

DETAILED DESCRIPTION

The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

In the proposed methods, an inter prediction offset is derived from one or multiple neighboring coded blocks.

The offset is then added to the motion compensation prediction block.

In the other aspect, the offset is used to predict the residual signal. In other words, the motion compensation prediction residual signal is subtracted by the inter prediction offset.

Let Orig_X and Pred_X be the original and motion compensation prediction signal of current block X, Offset_X be the derived inter prediction offset from the neighboring block. The final residual signal of current block is Resi_X＝Orig_X-Pred_X-Offset_X. At the decoder side, the reconstructed signal is Reco_X＝Resi'_X+Pred_X+Offset_X, where Resi'_X is the reconstructed residual signal.

In one embodiment, Offset_X＝mean (Resi'_Y) is derived as the mean value of Resi'_Y, where Resi'_Y is the reconstructed residual signal of the neighboring coded block Y.

In another embodiment, if an offset is already applied to the neighboring coded block, thenOffset_X＝mean (Resi'_Y) +Offset_Y, where Offset_Y is the inter prediction offset applied to the neighboring coded block.

In still another embodiment, the neighboring coded block used for inter prediction offset derivation can be a prediction block, transform block, 4x4 block or other predefined block sizes.

In still another embodiment, the used neighboring coded block is those that have same motion information as current block. In the other words, the motion vector (s) and reference frame (s) index of the used neighboring coded block should be equal to the motion vector (s) and reference frame (s) index of current block.

In still another embodiment, the neighboring coded block can be chosen from the left or the above. An example is shown in Fig. 1, where A and B are the neighboring coded blocks.

In still another embodiment, if multiple neighboring coded blocks are used, then the offset is calculated as the average. For example, if both A and B in Fig. 1 are used, then Offset_X＝ (mean (Resi'_A) +mean (Resi'_B) ) >>1. Another example is shown in Fig. 2, where the neighboring coded blocks can be chosen from A to C to B. The offset calculation can be described as follows:

where MV_curr represent the motion of current block, and MV_i represent a neighboring coded block i, and DC_i represents the mean value of the residual of block i. Note that the block i can be a 4x4 block or any other predefined block sizes.

In still another embodiment, the mean value of residual signal can be pre-calculated and stored. For example, when the residual of a block is reconstructed, then the mean value of this block is immediately obtained and stored in a buffer.

In still another embodiment, if current block is merge mode, then the neighboring block can be the same as the merge candidate. An example is shown in Fig. 3. If current block is merged to one of the spatial neighboring coded block, then the mean value of residual in that block is used as inter prediction offset.

In still another embodiment, Offset_X is set to zero if Offset_X is larger than a threshold.

In still another embodiment, Offset_X is set to zero if neighboring block is smaller than current block.

In still another embodiment, Offset_X is set to zero if the variance of neighboring block is larger than a threshold.

In still another embodiment, the inter prediction offset is applied in all components if the video is RGB, YUV, YCbCr or other format.

In still another embodiment, the inter prediction offset is only applied in luma component if the video is YUV or YCbCr format.

In still another embodiment, inter prediction offset can be constrained to applied for merge mode only.

In still another embodiment, inter prediction offset can be constrained to applied for merge 2Nx2N mode only.

In still another embodiment, inter prediction offset can be constrained to applied for 2Nx2N partition only.

In still another embodiment, inter prediction offset can be constrained to applied for other predefined partition size.

The methods described above can be used in a video encoder as well as in a video decoder. Embodiments of merge with inter prediction offset according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA) . These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art) . Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

Amethod of inter prediction offset, comprising:

the residual signal for current block is calculated as Resi_X＝Orig_X-Pred_X-Offset_X, where Orig_X is the original signal, Pred_Xis the prediction signal, Offset_Xis the inter prediction offset, at the decoder side, the reconstructed signal isReco_X＝Resi'_X+Pred_X+Offset_X, where Resi'_X is the reconstructed residual signal； and

the inter prediction offset Offset_X is derived from the residual signal of neighboring coded block (s) .
The method as claimed in claim 1, wherein the offset Offset_X＝mean (Resi'_Y) is derived as mean value of the reconstructed residual signal of a neighboring coded block Y.
The method as claimed in claim 1 and claim 2, wherein if an offset is already applied to the neighboring coded block Y, thenOffset_X＝mean (Resi'_Y) +Offset_Y, where Offset_Yis the inter prediction offset applied to the neighboring coded block.
The method as claimed in claim 2 and claim 3, wherein neighboring coded block Y can be a prediction block, transform block, 4x4 block or other predefined block sizes.
The method as claimed in claim 2 and claim 3, wherein Y is chosen from those that have same motion information as current block, in the other words, the motion vector (s) and reference frame (s) index of Y should be equal to the motion vector (s) and reference frame (s) index of current block X.
The method as claimed in claim 2 and claim 3, wherein Y can be chosen from the left or the above of current block X.
The method as claimed in claim 1, wherein multiple neighboring coded blocks are used to derive the offset, if both A and B are used, thenOffset_X＝ (mean (Resi'_A ) +mean (Resi'_B)) >>1； the neighboring coded blocks can be chosen from A to C to B, the offset is calculated by averaging over the used blocks:

where MV_curr represent the motion of current block, and MV_i represent a neighboring coded block i, and DC_i represents the mean value of the residual of block i, the block i can be a 4x4 block or any other predefined block sizes.
The method as claimed in claim 2 to claim 7, wherein the mean value of residual signal in a block can be pre-calculated and stored, when the residual of a block is reconstructed, then the mean value of this block is immediately obtained and stored in a buffer.
The method as claimed in claim 8, wherein the block used for pre-calculating the mean value can be a prediction unit (PU) , transform unit (TU) , 4x4 block or any predefined block size.
The method as claimed in claim 1, wherein if current block is merge mode, then the neighboring block can be the same as the merge candidate, if current block is merged to one of the spatial neighboring coded block, then the mean value of residual in that block is used as inter prediction offset.
The method as claimed in claim 1, wherein Offset_X is set to zero if Offset_Xis larger than a threshold.
The method as claimed in claim 1, wherein Offset_X is set to zero if neighboring block is smaller than current block.
The method as claimed in claim 1, wherein Offset_X is set to zero if the variance of neighboring block is larger than a threshold.
The method as claimed in claim 1, wherein the inter prediction offset is applied in all components if the video is RGB, YUV, YCbCr or other format.
The method as claimed in claim 1, wherein the inter prediction offset is only applied in luma component if the video is YUV or YCbCr format.
The method as claimed in claim 1, inter prediction offset can be constrained to applied for merge mode only.
The method as claimed in claim 1, inter prediction offset can be constrained to applied for 2Nx2N partition only.
The method as claimed in claim 1, inter prediction offset can be constrained to applied for other predefined partition size.
The method as claimed in claim 1, wherein one or more syntax elements can be used to signal whether merge with inter prediction offset is used, the syntax element can be coded.
The method as claimed in claim 19, wherein the syntax elements can be explicitly transmitted in the sequence level, view level, picture level, slice level, or other levels, it can be coded in VPS, SPS, PPS, APS, slice header, LCU et al.
The method as claimed in claim 20, wherein the information about whether the merge with inter prediction offset is used can also be derived implicitly at decoder side according to statistics of mode selections.