WO2019119910A1 - 一种用于p帧或b帧的帧内帧间组合预测方法 - Google Patents

一种用于p帧或b帧的帧内帧间组合预测方法 Download PDF

Info

Publication number
WO2019119910A1
WO2019119910A1 PCT/CN2018/107210 CN2018107210W WO2019119910A1 WO 2019119910 A1 WO2019119910 A1 WO 2019119910A1 CN 2018107210 W CN2018107210 W CN 2018107210W WO 2019119910 A1 WO2019119910 A1 WO 2019119910A1
Authority
WO
WIPO (PCT)
Prior art keywords
prediction
intra
frame
inter
prediction block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/107210
Other languages
English (en)
French (fr)
Inventor
王荣刚
范逵
李革
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to US16/629,777 priority Critical patent/US11051027B2/en
Publication of WO2019119910A1 publication Critical patent/WO2019119910A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock

Definitions

  • the present invention relates to the field of video coding technologies, and in particular, to a prediction method for an inter prediction frame (ie, a P frame or a B frame), which can combine an intra prediction block and an inter prediction block to obtain a final prediction block. Enhance the accuracy of the prediction to improve coding efficiency.
  • an inter prediction frame ie, a P frame or a B frame
  • intra prediction and inter prediction are very effective tools to remove redundant information in a video sequence.
  • inter prediction utilizes the correlation between the current frame and its reference frame to reduce time domain redundancy
  • intra prediction utilizes the similarity between spatially adjacent pixels to eliminate spatial redundancy.
  • Intra prediction is used for intra prediction frames (I frames) or intra prediction blocks in inter prediction frames
  • inter prediction can only be used in inter prediction frames (forward prediction frame P frames or bidirectional Predict frame B frame).
  • the existing intra prediction and inter prediction methods are independent of each other, and do not comprehensively consider the advantages and disadvantages of intra prediction and inter prediction.
  • only the optimal prediction mode is selected through the coding end decision, and the two prediction methods cannot be better utilized.
  • the advantage therefore, limits the performance of the forecast to a certain extent.
  • the present invention provides an intra-frame inter-frame combining prediction method for a P frame or a B frame, which is used to effectively reduce prediction block distortion and improve prediction accuracy of a prediction block.
  • the block and inter prediction block weighting method obtains the final prediction block; wherein the weighting coefficients of the intra prediction block and the inter prediction block are obtained according to the prediction distortion statistics of the two prediction methods; thereby, the prediction precision can be increased, and the prediction block is improved. Coding efficiency; includes the following steps:
  • each coding unit transmits a 1-bit tag to the code stream, and is used to identify whether an intra-frame inter-frame combination prediction method is used;
  • inter-frame prediction information is used to generate an inter-predicted block.
  • the inter prediction modes used are different, and the inter prediction information to be transmitted is also different. If the motion information of the inter prediction mode used in the intra-frame interframe combining prediction is derived by derivation, only the syntax element related to the derivation of the inter-frame information needs to be transmitted. If the motion information of the inter prediction mode used in the intra-frame interframe combining prediction is obtained by motion estimation, the corresponding motion information needs to be transmitted.
  • a new inter prediction mode can also be defined in the intra-frame interframe combining prediction, which is used in combination with the intra prediction block, as long as the corresponding inter prediction information is transmitted to the decoding end.
  • a 1-bit mark in the code stream is read by a coding unit at the decoding end, and decoded according to the mark; the following steps are performed:
  • intra-frame inter-frame combination prediction method the following operations are performed: first, inter-prediction information is read from the code stream, and inter-frame prediction is performed on the coding unit (CU), and then on-frame. The prediction and inter prediction are weighted averaged to obtain an intra-frame inter-frame prediction block. Finally, the reconstruction of the current coding block (intra-frame inter-frame prediction block) is performed.
  • the core of the present invention is a process of combining an intra prediction block and an inter prediction block, as shown in Equation 1:
  • P intra (x, y) and P inter (x, y) are pixel values of intra and inter prediction blocks, respectively, and W intra (x, y) is a weighting coefficient of intra prediction pixel values.
  • N is the current predicted block size.
  • the weight coefficients used are designed according to the sizes of intra-frame and inter-frame prediction distortion, the weight coefficients are inversely proportional to the prediction block distortion, and the distortion of the prediction block is obtained by statistics.
  • the present invention provides a prediction method for combining intra prediction and inter prediction, which can reduce the overall distortion of the prediction block and increase the prediction accuracy, thereby improving the coding efficiency of the prediction block.
  • the technical advantages of the present invention are embodied in the following aspects:
  • the present invention determines whether intra-frame interframe combining prediction is used by RDO decision. Since not every prediction block is suitable for intra-frame interframe combining, the practicality and robustness of this technique is increased by RDO decision.
  • the present invention proposes to use weighting coefficients which are inversely proportional to the prediction block distortion, and can not only make full use of the advantages of intra prediction and inter prediction, but also select the two methods to predict the optimal parts to be combined, and can be removed to some extent.
  • the intra prediction block and the inter prediction block are too distorted to obtain a better prediction effect.
  • FIG. 1 is a block diagram showing the encoding flow of a coding unit (CU) in the present invention.
  • FIG. 2 is a block diagram showing the decoding flow of a coding unit (CU) in the present invention.
  • FIG. 3 is a schematic diagram of an intra prediction mode in HEVC.
  • FIG. 4 is a schematic diagram of weighting coefficients in units of rows or columns in a 16 ⁇ 16 intra-frame inter-frame combination prediction block according to an embodiment of the present invention.
  • the invention provides an effective intra-frame inter-frame combination prediction method, which is adaptively selected by RDO decision whether to use the method; the proposed combined prediction method obtains a new prediction by weighting average of intra prediction block and inter prediction block
  • the weight coefficient used is designed according to statistical intra and inter prediction distortion, and the weight coefficient is inversely proportional to the predicted block distortion obtained by statistics; the invention can solve the prediction block distortion problem, and can increase the prediction precision and improve the coding efficiency of the prediction block. .
  • FIG. 1 is a flow chart of encoding at the encoding end of the present invention.
  • intra prediction and inter prediction are first performed. Then, the intra prediction block and the inter prediction block are weighted and averaged to obtain an intra intraframe combined prediction block.
  • RDO Rate-distortion optimization
  • RDO Rate-distortion optimization
  • the intra-frame inter-frame combining prediction method is used, the used inter-frame prediction information is encoded into the code stream.
  • FIG. 2 is a block diagram of a decoding process of a decoding end of the present invention.
  • the 1-bit flag is first read.
  • intra prediction is performed. Whether or not to perform the intra-frame combination prediction method is determined based on the value of the read flag. If the intra-frame inter-frame combination prediction is not used, the reconstruction of the coding block can be performed directly. Otherwise, the inter-prediction information is read from the code stream, and the coding unit (CU) is inter-predicted, and then the intra-frame and inter-frame prediction are weighted and averaged to obtain an intra-frame inter-frame prediction block. . Finally, the reconstruction of the current coding block is performed.
  • CU coding unit
  • the weight coefficients in the intra-frame inter-frame combination prediction method proposed by the present invention are designed according to intra-frame and inter-frame prediction distortion, and the distortion of the prediction block is obtained by statistics, and the weight coefficient is inversely proportional to the statistical prediction block distortion.
  • the design process of the weight coefficient is as follows:
  • the weighting coefficients of the intra prediction block and the inter prediction block may be expressed as Equations 2 and 3, respectively, and the sum of the two is 1.
  • W intra (x, y) and W inter (x, y) are weighting coefficients of the intra prediction block and the inter prediction block, respectively, and x and y are coordinates of the pixel points in the prediction block.
  • intra prediction blocks can be divided into four groups according to an intra prediction mode. Take the HEVC intra prediction mode as an example, as shown in Figure 3. The first group includes modes 0 and 1, the second group includes modes 2 to 13, the third group includes modes 14 to 22, and the fourth group includes modes 23 to 34. A set of weighting coefficients is counted for each group.
  • weighting coefficients in units of rows or columns may be used, that is, one row or one column of pixels in the prediction block uses the same weighting coefficient.
  • Figure 4 shows the weighting coefficients in rows or columns for a 16x16 intra prediction block design in a B frame.
  • (a), (b), and (c) of FIG. 4 are weighting coefficients of the second group, the third group, and the fourth group of intra prediction blocks, respectively.
  • the intra prediction block in the first group directly uses 1/2 as a weighting coefficient for the combination of prediction blocks.
  • the floating point weighting coefficients proposed in the present invention can be converted into integers, and right shifting operations are performed after weighting. Specifically, the weight coefficient can be multiplied by the power of m of 2, and the weighted value is then shifted to the right by m bits.
  • the expression after the conversion of Equation 1 into an integer is as shown in Equation 4.
  • the value of m is related to the required calculation accuracy. The higher the required precision, the larger the m value and the more accurate the calculation result.
  • inter-frame prediction information is used to generate an inter-predicted block.
  • the inter prediction modes used are different, and the inter prediction information to be transmitted is also different. If the motion information of the inter prediction mode used in the intra-frame interframe combining prediction is derived by deriving, for example, the skip mode and the merge mode in HEVC, only the syntax elements related to the derivation of the inter-frame information need to be transmitted. Specifically, if the skip mode is used in the intra-frame combination prediction, the information of the skip mode, that is, the index number of the skip mode motion information candidate needs to be transmitted; and if the merge mode is used, the merge mode motion information candidate needs to be transmitted. Index number and residual information.
  • the motion information of the inter prediction mode used in the intra-frame interframe combining prediction is obtained by motion estimation, the corresponding motion information needs to be transmitted.
  • a new inter prediction mode can also be defined in the intra-frame interframe combining prediction, which is used in combination with the intra prediction block, as long as the corresponding inter prediction information is transmitted to the decoding end.
  • the first motion information candidate of the skip mode can be fixedly used, and since a fixed inter prediction mode is used, it is not necessary to transmit any inter prediction information to the decoding end.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

一种用于P帧或B帧的帧内帧间组合预测的方法，通过率失真优化RDO决策自适应地选择是否使用所述帧内帧间组合预测；帧内帧间组合预测使用帧内预测块和帧间预测块加权的方法得到最终预测块；帧内预测块和帧间预测块的加权系数根据预测方法的预测失真统计得到。由此能够增加预测精度，提升预测块的编码和解码效率。本发明充分利用帧内预测和帧间预测的优势，选择这两种方法预测最优的部分组合在一起，可以一定程度上去除帧内预测块和帧间预测块失真过大的区域，获得更好的预测效果，实用性和鲁棒性俱佳。

Description

一种用于P帧或B帧的帧内帧间组合预测方法 技术领域
本发明涉及视频编码技术领域，具体涉及一种用于帧间预测帧(即P帧或是B帧)的预测方法，可将帧内预测块和帧间预测块相结合得到最终的预测块，增强预测的准确度，从而提升编码效率。
背景技术
在视频编码中，帧内预测和帧间预测是非常有效的工具，可以去除视频序列中的冗余信息。具体来说，帧间预测利用当前帧与其参考帧之间的相关性减小时域冗余，而帧内预测利用空间相邻像素点之间的相似性消除空域冗余。
在现在编码标准中，帧内预测和帧间预测相互独立。帧内预测被用于帧内预测帧(I帧)或是帧间预测帧中的帧内预测块，而帧间预测只能用在帧间预测帧中(前向预测帧P帧或是双向预测帧B帧)。
上述预测技术主要存在的缺陷是：
现有帧内预测和帧间预测方法相互独立，并没有综合考虑帧内预测和帧间预测的优势和缺陷，只是通过编码端决策选择最优的预测模式，无法更好的发挥两种预测方法的优势，因此在一定程度上局限了预测的性能。
发明内容
为了克服上述现有技术的不足，本发明提供一种用于P帧或B帧的帧内帧间组合预测方法，用以有效地减小预测块失真，提升预测块的预测精度。
本发明提供的技术方案是：
一种用于P帧或B帧的帧内帧间组合预测方法，通过率失真优化(Rate distortion optimization,RDO)决策自适应地选择是否使用该预测方法；本发明中的预测方法使用帧内预测块和帧间预测块加权的方式得到最终的预测块；其中帧内预测块和帧间预测块的加权系数根据两种预测方法的预测失真统计得到；由此能够增加预测精度，提升预测块的编码效率；包括如下步骤：
1)在编码端对每一个编码单元(Coding unit,CU)进行率失真优化，执行如下操作：
11)首先，对一个编码单元(Coding unit,CU)进行帧内预测
12)之后，对该编码单元(Coding unit,CU)进行帧间预测；
13)对帧内预测和帧间预测进行加权平均，得到帧内帧间组合预测块；
14)通过RDO决策确定是否使用帧内帧间组合预测方法；每一个编码单元中均传输1-bit的标记到码流中，用于标识是否使用了帧内帧间组合预测方法；
15)若使用了帧内帧间组合预测方法，则将所用到的帧间预测信息写入码流中。
在帧内帧间组合预测中，帧间预测信息用于产生帧间预测块。所使用的帧间预测模式不同，所需要传输的帧间预测信息也不同。如果在帧内帧间组合预测中使用的帧间预测模式的运动信息是通过导出得到的，则只需要传输导出帧间信息相关的语法元素。如果在帧内帧间组合预测中使用的帧间预测模式的运动信息是通过运动估计得出的，则需要传输相应的运动信息。在帧内帧间组合预测中也可以定义全新的帧间预测模式，用于与帧内预测块相结合，只要将相应的帧间预测信息传输到解码端即可。
2)在解码端一个编码单元读取码流中的1-bit标记，根据标记进行解码；执行如下步骤：
21)首先，对一个编码单元(Coding unit,CU)进行帧内预测
22)根据从码流中读取的1-bit标记确定解码端是否使用帧内帧间组合预测方法；
23)若使用了帧内帧间组合预测方法，则执行以下操作：首先从码流中读取帧间预测信息，并对此编码单元(Coding unit,CU)进行帧间预测，之后对帧内预测和帧间预测进行加权平均，得到帧内帧间组合预测块。最后进行当前编码块(帧内帧间组合预测块)的重建。
24)如果不使用帧内帧间组合预测方法，则直接进行编码块重建。
本发明的核心是对帧内预测块和帧间预测块组合的过程，如式1所示：
P′ comb(x,y)＝W intra(x,y)·P intra(x,y)+(1-W intra(x,y))·P inter(x,y)  (式1)
0≤x,y＜N
其中，P intra(x,y)和P inter(x,y)分别为帧内和帧间预测块的像素值，W intra(x,y)为帧内预测像素值的加权系数。P′ comb(x,y)为进行加权之后的帧内帧间预测组合的像素值；x、y为预测块内的坐标，x＝0表示预测块中的第一列，y＝0表示预测块中的第一行。N为当前预测块尺寸。
本发明提出的帧内帧间组合预测方法中，采用的权重系数根据帧内和帧间预测失真的大小设计，权重系数反比于预测块失真，而预测块的失真通过统计得到。
与现有技术相比，本发明的有益效果是：
本发明提供一种用于将帧内预测和帧间预测相结合的预测方法，能够减小预测块总体失真，增加预测精度，从而提升预测块的编码效率。具体地，本发明的技术优势体现为以下几方面：
本发明通过RDO决策决定是否使用帧内帧间组合预测。由于并不是每一个预测块都适合进行帧内帧间组合，所以通过RDO决策增加了此技术的实用性和鲁棒性。
并且，本发明提出使用与预测块失真成反比的加权系数，不仅可以充分利用帧内预测和帧间预测的优势，选择这两种方法预测最优的部分组合在一起，而且可以一定程度上去除帧内预测块和帧间预测块失真过大的区域，获得更好的预测效果。
附图说明
图1为本发明中的编码单元(CU)的编码流程框图。
图2为本发明中的编码单元(CU)的解码流程框图。
图3为HEVC中帧内预测模式示意图。
图4为本发明具体实施例中16×16的帧内帧间组合预测块中以行或列为单位的加权系数示意图。
具体实施方式
下面结合附图，通过实施例进一步描述本发明，但不以任何方式限制本发明的范围。
本发明提供一种有效的帧内帧间组合预测方法，通过RDO决策自适应的选择是否使用该方法；提出的组合预测方法，通过对帧内预测块和帧间预测块加权平均得到新的预测块；采用的权重系数根据统计的帧内和帧间预测失真设计，权重系数反比于统计得到的预测块失真；本发明能够解决预测块失真问题，并能够增加预测精度，提升预测块的编码效率。
图1为本发明编码端的编码流程图。对于一个编码单元来说，首先进行帧内预测和帧间预测。随后对帧内预测块和帧间预测块加权平均，得到帧内帧间组合预测块。之后使用率失真优化(RDO：Rate-distortion optimization)选择是否使用帧内帧间组合预测，并输出1-bit的标记到码流中。若使用了帧内帧间组合预测方法，则将所用到的帧间预测信息编码到码流中。
图2为本发明解码端的解码流程框图。对一个编码单元，首先读取1-bit的标记。随后进行帧内预测。根据读取的标记的值决定是否进行帧内帧间组合预测方法。如果不使用帧内帧间组合预测，则直接进行编码块的重建即可。否则，从码流中读取帧间预测信息，并对此编 码单元(Coding unit,CU)进行帧间预测，之后对帧内预测和帧间预测进行加权平均，得到帧内帧间组合预测块。最后进行当前编码块的重建。
本发明提出的帧内帧间组合预测方法中的权重系数根据帧内和帧间预测失真设计，预测块的失真通过统计得到，权重系数与所统计的预测块失真成反比。权重系数的设计过程如下：
首先，我们统计每一种帧内预测模式的预测失真分布，表示为D intra(x,y)，之后统计帧间预测块的预测失真为D inter(x,y)，则在帧内帧间组合预测中，帧内预测块和帧间预测块的加权系数可以分别表示为式2和式3，二者之和为1。
Figure PCTCN2018107210-appb-000001
其中，W intra(x,y)和W inter(x,y)分别为帧内预测块和帧间预测块的加权系数，x和y为预测块内像素点的坐标。
通过对编码结果的统计，可以对每一个预测块尺寸、每一个帧内预测模式都产生一组加权系数。另外，考虑到P帧和B帧中预测块失真可能存在不同，所以需要对P帧和B帧中的加权系数分开设计。
为了节约存储加权系数的空间，可对加权系数进行简化。例如，本发明中可将帧内预测块按照帧内预测模式分成4组。以HEVC帧内预测模式为例，如图3所示。其中第一组包括模式0和1，第二组包括模式2～13，第三组包括模式14～22，第四组为模式23～34。对每个分组统计一组加权系数。
另外，为了进一步节约存储加权系数的空间，可以使用以行或列为单位的加权系数，即预测块中的一行或是一列像素点使用同一个加权系数。例如在HEVC中我们可以对以上四组帧内预测模式分别设计以行或是列为单位的加权系数。图4给出了对B帧中16x16的帧内预测块设计的以行或是列为单位的加权系数。图4中(a)、(b)、(c)分别为第二组、第三组、第四组帧内预测块的加权系数。而第一组中的帧内预测块直接使用1/2作为加权系数进行预测块的组合。
为了避免进行浮点计算，可将本发明中所提出的浮点加权系数转换为整数，并在加权之后进行右移操作。具体来说，可将权重系数乘以2的m次幂，加权计算之后再将预测值右移m位。式1转化为整数计算后的表达式如式4所示。m的取值与所需的计算精度相关，所需的精度越高则m值越大，计算结果越准确。
P′ comb(x,y)＝(2 m·W intra(x,y)·P intra(x,y)+(2 m-2 m·W intra(x,y))·Ｐ inter(x,y)+2 m-1)＞＞m
                                                         (式4)
在帧内帧间组合预测中，帧间预测信息用于产生帧间预测块。所使用的帧间预测模式不同，所需要传输的帧间预测信息也不同。如果在帧内帧间组合预测中使用的帧间预测模式的运动信息是通过导出得到的，例如HEVC中的skip模式、merge模式，则只需要传输导出帧间信息相关的语法元素。具体地说，如果在帧内帧间组合预测中使用了skip模式，则需要传输skip模式的信息，即skip模式运动信息候选的索引号；而如果使用merge模式，则需要传输merge模式运动信息候选的索引号以及残差信息。如果在帧内帧间组合预测中使用的帧间预测模式的运动信息是通过运动估计得出的，则需要传输相应的运动信息。在帧内帧间组合预测中也可以定义全新的帧间预测模式，用于与帧内预测块相结合，只要将相应的帧间预测信息传输到解码端即可。例如可以固定使用skip模式的第一个运动信息候选，由于使用了固定的帧间预测模式，所以无需传输任何帧间预测信息到解码端。
需要注意的是，公布实施例的目的在于帮助进一步理解本发明，但是本领域的技术人员可以理解：在不脱离本发明及所附权利要求的精神和范围内，各种替换和修改都是可能的。因此，本发明不应局限于实施例所公开的内容，本发明要求保护的范围以权利要求书界定的范围为准。

Claims (10)

  1. 一种用于帧内帧间组合预测的编码方法，通过率失真优化决策自适应地选择是否使用所述帧内帧间组合预测；帧内帧间组合预测使用帧内预测块和帧间预测块加权的方法得到最终预测块；帧内预测块和帧间预测块的加权系数根据预测方法的预测失真统计得到；
    针对P帧或B帧的帧内帧间组合预测，首先在编码端对每一个编码单元进行率失真优化，执行如下操作：
    11)对一个编码单元进行帧内预测；
    12)对该编码单元进行帧间预测；
    13)对帧内预测和帧间预测进行加权平均，得到帧内帧间组合预测块；
    14)通过率失真优化决策确定是否使用帧内帧间组合预测方法；每一个编码单元中均传输一个组合预测标记到码流中，用于标识是否使用了帧内帧间组合预测方法；
    15)所述帧内帧间组合预测方法表示为式1：
    P′ comb(x,y)＝W intra(x,y)·P intra(x,y)+(1-W intra(x,y))·P inter(x,y)   (式1)
    0≤x,y<N
    其中，P intra(x,y)和P inter(x,y)分别为帧内和帧间预测块的像素值，W intra(x,y)为帧内预测像素值的加权系数；P′ comb(x,y)为进行加权之后的帧内帧间预测组合的像素值；x、y为预测块内的坐标，x＝0表示预测块中的第一列，y＝0表示预测块中的第一行；N为当前预测块尺寸；
    若使用了帧内帧间组合预测方法，则将帧间预测信息写入码流中，完成帧内帧间组合预测的编码过程。
  2. 如权利要求1所述用于帧内帧间组合预测的编码方法，其特征是，所述组合预测标记为1-bit的标记。
  3. 如权利要求1所述用于帧内帧间组合预测的编码方法，其特征是，根据帧内和帧间预测失真大小采用不同的加权系数，加权系数与预测块的失真成反比；预测块的失真通过统计得到，具体统计方法是：
    对每一种帧内预测模式的预测失真分布进行统计，表示为D intra(x,y)；
    对帧间预测块的预测失真进行统计，表示为D inter(x,y)；
    在帧内帧间组合预测中，帧内预测块和帧间预测块的加权系数之和为1，分别表示为式2和式3：
    Figure PCTCN2018107210-appb-100001
    Figure PCTCN2018107210-appb-100002
    其中，W intra(x,y)和W inter(x,y)分别为帧内预测块和帧间预测块的加权系数，x和y为预测块内像素点的坐标；
    对每一个预测块尺寸、每一个帧内预测模式均分别产生一组加权系数；并对P帧和B帧分别产生不同的加权系数。
  4. 如权利要求3所述用于帧内帧间组合预测的编码方法，其特征是，对加权系数进行简化或使用以行或列为单位的加权系数，以节约存储加权系数的空间。
  5. 如权利要求3所述用于帧内帧间组合预测的编码方法，其特征是，将浮点加权系数转换为整数，并在加权之后进行右移操作，以避免进行浮点计算；
    具体地，将加权系数乘以2的m次幂，加权计算之后再将预测值右移m位；将式1转化为整数计算后表示为式4：
    P′ comb(x,y)＝(2 m·W intra(x,y)·P intra(x,y)+(2 m-2 m·W intra(x,y))·P inter(x,y)+
    2 m-1)＞＞m
                                   (式4)
    其中，m的取值与计算精度相关，所需的精度越高则m值越大，计算结果越准确。
  6. 一种用于帧内帧间组合预测的解码方法，通过率失真优化(RDO)决策自适应地选择是否使用所述帧内帧间组合预测；帧内帧间组合预测使用帧内预测块和帧间预测块加权的方法得到最终预测块；帧内预测块和帧间预测块的加权系数根据预测方法的预测失真统计得到；
    针对P帧或B帧的帧内帧间组合预测，在解码端对每一个编码单元读取码流中的组合预测标记，根据标记进行解码；执行如下步骤：
    21)对编码单元进行帧内预测；
    22)根据从码流中读取的组合预测标记，确定解码端是否使用帧内帧间组合预测方法；
    23)若使用了帧内帧间组合预测方法，则执行以下操作：首先从码流中读取帧间预测信息，并对此编码单元进行帧间预测，之后对帧内预测和帧间预测进行加权平均，得到帧内帧 间组合预测块；最后进行当前编码块的重建；
    所述帧内帧间组合预测块具体通过式1进行预测得到：
    P′ comb(x,y)＝W intra(x,y)·P intra(x,y)+(1-W intra(x,y))·P inter(x,y)   (式1)
    0≤x,y<N
    其中，P intra(x,y)和P inter(x,y)分别为帧内和帧间预测块的像素值，W intra(x,y)为帧内预测像素值的加权系数；P′ comb(x,y)为进行加权之后的帧内帧间预测组合的像素值；x、y为预测块内的坐标，x＝0表示预测块中的第一列，y＝0表示预测块中的第一行；N为当前预测块尺寸；
    24)如果不使用帧内帧间组合预测方法，则直接进行编码块重建。
  7. 如权利要求6所述用于帧内帧间组合预测的解码方法，其特征是，组合预测标记为1-bit的标记。
  8. 如权利要求6所述用于帧内帧间组合预测的解码方法，其特征是，根据帧内和帧间预测失真大小采用不同的加权系数，加权系数与预测块的失真成反比；预测块的失真通过统计得到，具体统计方法是：
    对每一种帧内预测模式的预测失真分布进行统计，表示为D intra(x,y)；
    对帧间预测块的预测失真进行统计，表示为D inter(x,y)；
    在帧内帧间组合预测中，帧内预测块和帧间预测块的加权系数之和为1，分别表示为式2和式3：
    Figure PCTCN2018107210-appb-100003
    Figure PCTCN2018107210-appb-100004
    其中，W intra(x,y)和W inter(x,y)分别为帧内预测块和帧间预测块的加权系数，x和y为预测块内像素点的坐标；
    对每一个预测块尺寸、每一个帧内预测模式均分别产生一组加权系数；并对P帧和B帧分别产生不同的加权系数。
  9. 如权利要求8所述用于帧内帧间组合预测的解码方法，其特征是，对加权系数进行简化或使用以行或列为单位的加权系数，以节约存储加权系数的空间。
  10. 如权利要求8所述用于帧内帧间组合预测的解码方法，其特征是，将浮点加权系数 转换为整数，并在加权之后进行右移操作，以避免进行浮点计算；
    具体地，将加权系数乘以2的m次幂，加权计算之后再将预测值右移m位；将式5转化为整数计算后表示为式4：
    P′ comb(x,y)＝(2 m·W intra(x,y)·P intra(x,y)+(2 m-2 m·W intra(x,y))·P inter(x,y)+
    2 m-1)＞＞m
                       (式4)
    其中，m的取值与计算精度相关，所需的精度越高则m值越大，计算结果越准确。
PCT/CN2018/107210 2017-12-20 2018-09-25 一种用于p帧或b帧的帧内帧间组合预测方法 Ceased WO2019119910A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/629,777 US11051027B2 (en) 2017-12-20 2018-09-25 Intra-frame and inter-frame combined prediction method for P frames or B frames

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711381157.3A CN107995489A (zh) 2017-12-20 2017-12-20 一种用于p帧或b帧的帧内帧间组合预测方法
CN201711381157.3 2017-12-20

Publications (1)

Publication Number Publication Date
WO2019119910A1 true WO2019119910A1 (zh) 2019-06-27

Family

ID=62039172

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/107210 Ceased WO2019119910A1 (zh) 2017-12-20 2018-09-25 一种用于p帧或b帧的帧内帧间组合预测方法

Country Status (3)

Country Link
US (1) US11051027B2 (zh)
CN (1) CN107995489A (zh)
WO (1) WO2019119910A1 (zh)

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107995489A (zh) * 2017-12-20 2018-05-04 北京大学深圳研究生院 一种用于p帧或b帧的帧内帧间组合预测方法
KR102822660B1 (ko) 2018-04-04 2025-06-18 닛폰 호소 교카이 예측 화상 보정 장치, 화상 부호화 장치, 화상 복호 장치, 및 프로그램
CN108632616B (zh) * 2018-05-09 2021-06-01 电子科技大学 一种基于参考质量做帧间加权预测的方法
CN121309802A (zh) * 2018-06-18 2026-01-09 世宗大学校产学协力团 影像解码/编码方法及传输比特流的方法
CN111083484B (zh) 2018-10-22 2024-06-28 北京字节跳动网络技术有限公司 基于子块的预测
CN111083489B (zh) 2018-10-22 2024-05-14 北京字节跳动网络技术有限公司 多次迭代运动矢量细化
CN111436227B (zh) * 2018-11-12 2024-03-29 北京字节跳动网络技术有限公司 在视频处理中使用组合帧间-帧内预测
BR112021009149A2 (pt) * 2018-11-15 2021-08-10 Telefonaktiebolaget Lm Ericsson (Publ) método e aparelho para desbloquear uma fronteira, programa de computador, e, portadora
WO2020103852A1 (en) 2018-11-20 2020-05-28 Beijing Bytedance Network Technology Co., Ltd. Difference calculation based on patial position
WO2020103877A1 (en) 2018-11-20 2020-05-28 Beijing Bytedance Network Technology Co., Ltd. Coding and decoding of video coding modes
EP3857896A4 (en) 2018-11-22 2021-12-01 Beijing Bytedance Network Technology Co. Ltd. COORDINATION PROCESS FOR INTER-PREDICTION BASED ON SUB-BLOCKS
CN111294590A (zh) * 2018-12-06 2020-06-16 华为技术有限公司 用于多假设编码的加权预测方法及装置
CN111372086B (zh) * 2018-12-26 2021-08-03 华为技术有限公司 视频图像解码方法及装置
CN111385569B (zh) * 2018-12-28 2022-04-26 杭州海康威视数字技术股份有限公司 一种编解码方法及其设备
CN111010578B (zh) * 2018-12-28 2022-06-24 北京达佳互联信息技术有限公司 一种帧内帧间联合预测的方法、装置以及存储介质
CN109714596A (zh) * 2019-01-30 2019-05-03 江苏允博信息科技有限公司 一种基于深度学习的hevc帧内预测编码的方法
KR102696718B1 (ko) 2019-02-01 2024-08-21 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 조합된 인트라-인터 예측의 확장된 적용
US11290726B2 (en) 2019-02-07 2022-03-29 Qualcomm Incorporated Inter-intra prediction mode for video data
KR102635518B1 (ko) 2019-03-06 2024-02-07 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 변환된 단예측 후보의 사용
JP7267785B2 (ja) * 2019-03-11 2023-05-02 キヤノン株式会社 画像復号装置、画像復号方法、及びプログラム
MX2021010980A (es) 2019-03-21 2021-10-13 Beijing Bytedance Network Tech Co Ltd Procesamiento de ponderacion mejorado de intra-interprediccion combinadas.
GB2582929A (en) * 2019-04-08 2020-10-14 Canon Kk Residual signalling
CN113728640B (zh) 2019-04-23 2024-08-02 北京字节跳动网络技术有限公司 帧内预测和残差编解码
WO2020244536A1 (en) 2019-06-03 2020-12-10 Beijing Bytedance Network Technology Co., Ltd. Combined intra and intra-block copy prediction for video coding
CN120416462A (zh) * 2019-06-19 2025-08-01 Lg 电子株式会社 基于运动预测的图像编译方法和装置
CN113875251B (zh) * 2019-06-21 2023-11-28 华为技术有限公司 用于几何分割模式的自适应滤波器强度指示
CN114287131B (zh) 2019-08-19 2025-05-23 北京字节跳动网络技术有限公司 基于计数器的帧内预测模式的更新
WO2021047540A1 (en) 2019-09-09 2021-03-18 Beijing Bytedance Network Technology Co., Ltd. Coefficient scaling for high precision image and video coding
CN114731392B (zh) 2019-09-21 2025-01-03 北京字节跳动网络技术有限公司 用于图像和视频编解码的高精度变换和量化
WO2021052507A1 (en) * 2019-09-22 2021-03-25 Beijing Bytedance Network Technology Co., Ltd. Sub-picture coding and decoding of video
CN112543323B (zh) * 2019-09-23 2022-05-31 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
CN113709501B (zh) * 2019-12-23 2022-12-23 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
CN116711304A (zh) * 2020-12-28 2023-09-05 Oppo广东移动通信有限公司 预测方法、编码器、解码器以及存储介质
CN113794885B (zh) * 2020-12-30 2022-12-23 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
KR20230144056A (ko) * 2021-03-19 2023-10-13 캐논 가부시끼가이샤 부호화 장치 및 방법
JP2024523675A (ja) * 2021-07-02 2024-06-28 ヒョンダイ モーター カンパニー ビデオ符号化／復号化方法及び装置
CN120151513A (zh) * 2022-01-07 2025-06-13 杭州海康威视数字技术股份有限公司 图像编解码方法、装置及存储介质
CN115002486B (zh) * 2022-05-26 2025-06-03 百果园技术(新加坡)有限公司 编码单元预测块的权重确定方法及装置
CN115118977B (zh) * 2022-08-29 2022-11-04 华中科技大学 针对360度视频的帧内预测编码方法、系统及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101877785A (zh) * 2009-04-29 2010-11-03 祝志怡 一种基于混合预测的视频编码方法
US9609343B1 (en) * 2013-12-20 2017-03-28 Google Inc. Video coding using compound prediction
CN107113425A (zh) * 2014-11-06 2017-08-29 三星电子株式会社 视频编码方法和设备以及视频解码方法和设备
CN107995489A (zh) * 2017-12-20 2018-05-04 北京大学深圳研究生院 一种用于p帧或b帧的帧内帧间组合预测方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101877785A (zh) * 2009-04-29 2010-11-03 祝志怡 一种基于混合预测的视频编码方法
US9609343B1 (en) * 2013-12-20 2017-03-28 Google Inc. Video coding using compound prediction
CN107113425A (zh) * 2014-11-06 2017-08-29 三星电子株式会社 视频编码方法和设备以及视频解码方法和设备
CN107995489A (zh) * 2017-12-20 2018-05-04 北京大学深圳研究生院 一种用于p帧或b帧的帧内帧间组合预测方法

Also Published As

Publication number Publication date
CN107995489A (zh) 2018-05-04
US11051027B2 (en) 2021-06-29
US20200314432A1 (en) 2020-10-01

Similar Documents

Publication Publication Date Title
WO2019119910A1 (zh) 一种用于p帧或b帧的帧内帧间组合预测方法
CN107896330B (zh) 一种用于帧内和帧间预测的滤波方法
JP5081305B2 (ja) フレーム間予測符号化の方法および装置
CN101884219B (zh) 处理视频信号的方法和装置
CN102067601B (zh) 视频编码和解码中模板匹配预测(tmp)的方法和装置
KR101218434B1 (ko) 움직임 추정 방법, 시스템 및 컴퓨터 판독가능 매체
WO2020134968A1 (zh) 一种编解码方法及其设备
CN118921484A (zh) 图像编码/解码设备和发送图像数据的设备
CN102740077B (zh) 基于h.264/avc标准的帧内预测模式选择方法
CN111133759B (zh) 编码或解码视频数据的方法和装置
CN102077599B (zh) 用于视频编码器中的高质量帧内模式预测的设备和方法
WO2015010319A1 (zh) 一种基于p帧的多假设运动补偿编码方法
CN104488271A (zh) 一种基于p帧的多假设运动补偿方法
CN102186081B (zh) 一种基于梯度向量的h.264帧内模式选择方法
CN107087200A (zh) 针对高效率视频编码标准的跳过编码模式提前判决方法
CN102752588A (zh) 使用空间缩放预测的视频编解码方法
CN114650418A (zh) 解码方法、编码方法、装置及设备
WO2018120797A1 (zh) 一种用于帧间预测帧中的帧内预测方法
CN103327327A (zh) 用于高性能视频编码hevc的帧间预测编码单元选择方法
CN109714596A (zh) 一种基于深度学习的hevc帧内预测编码的方法
WO2020108360A1 (zh) 视频图像编解码中的运动信息确定
CN102611891B (zh) 一种在变换域内直接进行转换编码的方法
WO2021031225A1 (zh) 一种运动矢量导出方法、装置及电子设备
CN114710661B (zh) 一种解码、编码方法、装置及其设备
CN112449180B (zh) 一种编解码方法、装置及其设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18892704

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18892704

Country of ref document: EP

Kind code of ref document: A1