WO2019153861A1 - 一种双向帧间预测方法及装置 - Google Patents

一种双向帧间预测方法及装置 Download PDF

Info

Publication number
WO2019153861A1
WO2019153861A1 PCT/CN2018/119430 CN2018119430W WO2019153861A1 WO 2019153861 A1 WO2019153861 A1 WO 2019153861A1 CN 2018119430 W CN2018119430 W CN 2018119430W WO 2019153861 A1 WO2019153861 A1 WO 2019153861A1
Authority
WO
WIPO (PCT)
Prior art keywords
image block
template
weight value
reference image
preset
Prior art date
Application number
PCT/CN2018/119430
Other languages
English (en)
French (fr)
Inventor
毛珏
虞露
赵寅
杨海涛
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2019153861A1 publication Critical patent/WO2019153861A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding

Definitions

  • the embodiments of the present invention relate to the field of video image coding and decoding technologies, and in particular, to a bidirectional inter prediction method and apparatus.
  • motion-compensated inter-frame prediction techniques play a very important role in improving coding efficiency.
  • the predicted image block of the current image block may be generated only according to one reference image block (ie, unidirectional prediction), or the predicted image block of the current image block may be generated according to the two reference image blocks (ie, the bidirectional inter-frame Predicting), the above two reference image blocks may be from the same reference frame or different reference frames.
  • two reference image blocks are weighted according to respective weight values to generate a predicted image block of the current image block.
  • bidirectional inter prediction is to weight two reference image blocks according to the weight value of the strip level to generate a prediction image block, or use a default weight value (1/2, 1/2).
  • the two reference image blocks are uniformly weighted to generate a predicted image block.
  • the prediction image block generated using the existing method has a lower accuracy and lower prediction performance.
  • the embodiment of the present invention provides a bidirectional inter prediction method and device, which can solve the problem that the accuracy of the currently obtained predicted image block is low and the prediction performance is low.
  • a bidirectional inter prediction method is provided, and the bidirectional inter prediction method is:
  • the current template includes a pixel region adjacent to the current image block and reconstructed
  • the first reference template includes a pixel region adjacent to the first reference image block and reconstructed
  • a second reference template the second reference template includes a pixel region adjacent to the second reference image block and reconstructed
  • the relationship between the current template and the current image block and the relationship between the reference template and the reference image block are the same; then, calculating a first difference value between the current template and the first reference template, and between the current template and the second reference template a second difference value, and determining a difference between the degree of difference between the first difference and the second difference value, and determining a weight value corresponding to the first reference image block and the second reference image according to the preset range of the difference attribution
  • the weight value corresponding to the block so that the first reference image block can be obtained according to the weight value corresponding to the first
  • the bidirectional inter prediction method provided by the embodiment of the present application determines the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, that is, the weight value of the block level is determined, compared to the prior art.
  • the bidirectional inter prediction method provided by the embodiment of the present application effectively improves the similarity between the current image block and the predicted image block, and the prediction performance is high.
  • the foregoing method for determining a gap is: determining a gap according to the following formula:
  • D represents the difference
  • S1 represents the second difference value
  • S0 represents the first difference value
  • n is the first preset constant
  • k is the second preset constant.
  • the foregoing method for determining a gap is: determining a gap according to the following formula:
  • D represents the difference
  • S1 represents the second difference value
  • S0 represents the first difference value
  • m is the third preset constant.
  • W0 is a weight value corresponding to the first reference image block
  • w1 is a weight value corresponding to the second reference image block
  • ⁇ w is a weight value offset
  • floor( ⁇ ) is a downward rounding function
  • P is a precision of the weight value.
  • o is the fourth preset constant
  • q is the fifth preset constant
  • i is the sixth preset constant
  • Dmin represents the minimum value of the gap D.
  • the bidirectional motion information of the current image block includes a first reference frame index for indicating a first reference frame, and a second for indicating a second reference frame.
  • the reference frame index, the first motion vector, and the second motion vector is included in another possible implementation manner of the present application.
  • the method of “determining the first reference image block and the second reference image block according to the bidirectional motion information of the current image block” is: acquiring the first reference frame index, the second reference frame index, and the first motion. And a second motion vector; obtaining, according to the first reference frame index, a first reference frame from the first reference frame list, and acquiring a second reference frame from the second reference frame list according to the second reference frame index; The motion vector and the first reference frame determine a first reference image block, and determine a second reference image block according to the second motion vector and the second reference frame.
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a fourth for indicating the second reference frame.
  • the reference frame index, the third motion vector, and the fourth motion vector is included in another possible implementation manner of the present application.
  • the method of “determining the first reference image block and the second reference image block according to the bidirectional motion information of the current image block” is: acquiring the third reference frame index, the fourth reference frame index, and the third motion. a vector and a fourth motion vector; determining, according to the third reference frame index and the third motion vector, the first image block from the first reference frame, and from the second reference frame according to the fourth reference frame index and the fourth motion vector Determining a second image block; acquiring the first target template and the second target template, where the first target template includes adjacent and reconstructed pixel regions of the first image block, and the second target template includes second image block adjacent And the reconstructed pixel area; determining, according to the difference value between the first target template and the second target template and the current template, the weight value corresponding to the first image block and the weight value corresponding to the second image block; Weighting the image block and the weight value corresponding to the second image block, weighting the pixels of the first image block and the pixels of the second image block to obtain a search template, and
  • the present application can directly determine the first reference image block and the second reference image block according to the obtained bidirectional motion information, and can also determine the first image block and the second image block by using the obtained bidirectional motion information, and Determining a search template according to the first image block and the second image block, and determining a first reference image block from the first reference frame according to the search template, and determining a second reference image block from the second reference frame. Determining the first reference image block and the second reference image block by using the search template further improves the degree of similarity between the current image block and the predicted image block.
  • the method for calculating the weight value corresponding to the two image blocks is: calculating a third difference value between the current template and the first target template, and a fourth difference value between the current template and the second target template, and according to the third difference value And the fourth difference value, determining a weight value corresponding to the first image block and a weight value corresponding to the second image block.
  • the method for the weight value corresponding to the two image blocks is: according to the preset weight value of N(N ⁇ 2), the pixels of the first target template and the pixels of the second target template are respectively weighted according to each pair of preset weight values.
  • the weight value determines a weight value corresponding to the first image block and a weight value corresponding to the second image block.
  • the weight value corresponding to the first image block and the weight value corresponding to the second image block may be determined by using different methods, which is not specifically limited in this embodiment of the present application.
  • a bidirectional inter prediction apparatus comprising a determining unit, an obtaining unit, a calculating unit, and a weighting unit.
  • the determining unit is configured to determine the first reference image block and the second reference image block according to the bidirectional motion information of the current image block.
  • the acquiring unit is configured to obtain a current template, a first reference template, and a second reference template, where the current template includes a pixel region adjacent to the current image block determined by the determining unit, and the reconstructed pixel region, where the first reference template includes the foregoing determination a unit-determined first reference image block adjacent and reconstructed pixel region, the second reference template includes a pixel region adjacent to the second reference image block determined by the determining unit and reconstructed; the current template and the current image block
  • the positional relationship, the positional relationship between the first reference template and the first reference image block, and the positional relationship of the second reference template and the second reference image block are the same.
  • the calculating unit is configured to calculate a first difference value between the current template and the first reference template acquired by the acquiring unit, and a second difference value between the current template and the second reference template.
  • the determining unit is further configured to determine a gap, where the difference is used to represent a degree of difference between the first difference value and the second difference value calculated by the calculating unit, and determine the first reference image block corresponding according to the preset range of the gap attribution.
  • the weight value and the weight value corresponding to the second reference image block are different.
  • the weighting unit is configured to perform, according to the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, by the determining unit, the pixels of the first reference image block and the pixels of the second reference image block. Weighting, the pixels of the predicted image block are obtained.
  • W0 is a weight value corresponding to the first reference image block
  • w1 is a weight value corresponding to the second reference image block
  • ⁇ w is a weight value offset
  • floor( ⁇ ) is a downward rounding function
  • P is a precision of the weight value.
  • o is the fourth preset constant
  • q is the fifth preset constant
  • i is the sixth preset constant
  • Dmin represents the minimum value of the gap D.
  • the bidirectional motion information of the current image block includes a first reference frame index for indicating a first reference frame, and a second for indicating a second reference frame.
  • the reference frame index, the first motion vector, and the second motion vector is included in another possible implementation manner of the present application.
  • the acquiring unit is further configured to acquire a first reference frame index, a second reference frame index, a first motion vector, and a second motion vector, and obtain the first reference frame list according to the first reference frame index. a reference frame, and acquiring a second reference frame from the second reference frame list according to the second reference frame index.
  • the determining unit is configured to determine a first reference image block according to the first motion vector and the first reference frame acquired by the acquiring unit, and determine a second reference image block according to the second motion vector and the second reference frame.
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a fourth for indicating the second reference frame.
  • the reference frame index, the third motion vector, and the fourth motion vector is included in another possible implementation manner of the present application.
  • the acquiring unit is further configured to acquire a third reference frame index, a fourth reference frame index, a third motion vector, and a fourth motion vector, and according to the third reference frame index and the third motion vector, from the first reference. Determining a first image block in the frame, and determining a second image block from the second reference frame according to the fourth reference frame index and the fourth motion vector, and acquiring the first target template and the second target template, the first target template including A pixel region adjacent to the first image block and reconstructed, and the second target template includes a pixel region adjacent to the second image block and reconstructed.
  • the determining unit is configured to determine, according to the difference value between the first target template and the second target template respectively obtained by the acquiring unit and the current template, the weight value corresponding to the first image block and the weight corresponding to the second image block. value.
  • the weighting unit is further configured to weight the pixels of the first image block and the pixels of the second image block according to the weight value of the first image block determined by the determining unit and the weight value corresponding to the second image block.
  • Search for a template The determining unit is configured to determine, according to the search template obtained by the weighting unit, a first reference image block from the first reference frame, and determine a second reference image block from the second reference frame.
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a second reference frame for indicating the second reference frame.
  • the calculating unit is further configured to calculate a third difference value between the current template and the first target template, and the current template and the second target template. The fourth difference between the values.
  • the determining unit is configured to determine, according to the third difference value and the fourth difference value calculated by the calculating unit, the weight value corresponding to the first image block and the weight value corresponding to the second image block.
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a second reference frame for indicating the second reference frame.
  • the weighting unit is further configured to: according to the N pair preset weight value, the pixels of the first target template and the pixels of the second target template are respectively A pair of preset weight values are weighted to obtain N composite space templates, N ⁇ 2.
  • the calculating unit is further configured to calculate a difference value between the current template and each of the synthesized spatial templates obtained by the weighting unit.
  • the determining unit is specifically configured to determine a composite space template with the smallest difference value calculated by the calculating unit, and determine a weight value corresponding to the first image block and the second image block according to a pair of preset weight values corresponding to the composite space template. The corresponding weight value.
  • a terminal comprising: one or more processors, a memory, and a communication interface.
  • the memory, communication interface is coupled to one or more processors; the memory is for storing computer program code, the computer program code comprising instructions, and when the one or more processors execute the instructions, the terminal performs the first aspect as described above and any one thereof A bidirectional inter-frame prediction method as described in a possible implementation.
  • the processor is configured to: determine a first reference image block and a second reference image block according to the bidirectional motion information of the current image block; acquire the current template, the first reference template, and the second reference template, the current template by using the communication interface A pixel region adjacent to the current image block and reconstructed, the first reference template includes a pixel region adjacent to the first reference image block and reconstructed, and the second reference template includes adjacent to the second reference image block, And the reconstructed pixel area, the positional relationship between the current template and the current image block, the positional relationship between the first reference template and the first reference image block, and the positional relationship between the second reference template and the second reference image block; a first difference value between the template and the first reference template, and a second difference value between the current template and the second reference template; determining a gap, the difference is used to represent the first difference value calculated by the calculating unit and the first Determining the difference degree of the difference value, and determining the weight value corresponding to the first reference image block and the second according to the preset
  • the weight value corresponding to the first reference image block is different, and the weight value corresponding to the second reference image block is different; according to the weight value corresponding to the first reference image block. And weighting the pixels of the first reference image block and the pixels of the second reference image block to obtain pixels of the predicted image block.
  • W0 is a weight value corresponding to the first reference image block
  • w1 is a weight value corresponding to the second reference image block
  • ⁇ w is a weight value offset
  • floor( ⁇ ) is a downward rounding function
  • P is a precision of the weight value.
  • o is the fourth preset constant
  • q is the fifth preset constant
  • i is the sixth preset constant
  • Dmin represents the minimum value of the gap D.
  • the bidirectional motion information of the current image block includes a first reference frame index for indicating a first reference frame, and a second for indicating a second reference frame.
  • the reference frame index, the first motion vector, and the second motion vector is included in another possible implementation manner of the present application.
  • the processor is further configured to: acquire a first reference frame index, a second reference frame index, a first motion vector, and a second motion vector, and obtain a first reference frame list according to the first reference frame index. a reference frame, and acquiring a second reference frame from the second reference frame list according to the second reference frame index; determining, according to the acquired first motion vector and the first reference frame, the first reference image block, and according to the second The motion vector and the second reference frame determine a second reference image block.
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a fourth for indicating the second reference frame.
  • the reference frame index, the third motion vector, and the fourth motion vector is included in another possible implementation manner of the present application.
  • the processor is further configured to: acquire a third reference frame index, a fourth reference frame index, a third motion vector, and a fourth motion vector, and according to the third reference frame index and the third motion vector, from the first reference Determining a first image block in the frame, and determining a second image block from the second reference frame according to the fourth reference frame index and the fourth motion vector, and acquiring the first target template and the second target template, the first target template including a pixel region adjacent to the first image block and reconstructed, the second target template includes a pixel region adjacent to the second image block and reconstructed; respectively, according to the obtained first target template and the second target template respectively a difference value between the first image block and a weight value corresponding to the second image block; and the first image block according to the weight value of the first image block and the weight value corresponding to the second image block
  • the pixels and the pixels of the second image block are weighted to obtain a search template; the first reference image block is determined from the first reference frame according to the search template, and the
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a second reference frame for indicating the second reference frame.
  • the processor is further configured to: calculate a third difference value between the current template and the first target template, and the current template and the second target template And a fourth difference value; and determining, according to the calculated third difference value and the fourth difference value, a weight value corresponding to the first image block and a weight value corresponding to the second image block.
  • the bidirectional motion information of the current image block includes a third reference frame index for indicating the first reference frame, and a second reference frame for indicating the second reference frame.
  • the processor is further configured to: according to the N pairs of preset weight values, the pixels of the first target template and the pixels of the second target template are respectively Weighting a pair of preset weight values to obtain N composite space templates, N ⁇ 2; calculating a difference value between the current template and each composite space template; determining a calculated composite space template having the smallest difference value, according to The pair of preset weight values corresponding to the composite space template determines a weight value corresponding to the first image block and a weight value corresponding to the second image block.
  • a fourth aspect provides a video encoder including a nonvolatile storage medium and a central processing unit, the nonvolatile storage medium storing an executable program, the central processing unit and the nonvolatile storage The medium is coupled and the executable program is executed to implement the bi-directional inter-frame prediction method of the first aspect and any one of its possible implementations.
  • a video decoder including a non-volatile storage medium and a central processing unit, the non-volatile storage medium storing an executable program, the central processing unit and the non-volatile storage The medium is coupled and the executable program is executed to implement the bi-directional inter-frame prediction method of the first aspect and any one of its possible implementations.
  • an encoder comprising the bidirectional inter prediction apparatus and the reconstruction module in the second aspect, wherein the reconstruction module is configured to generate a prediction image according to the bidirectional inter prediction apparatus The pixels of the block determine the reconstructed pixel value of the current image block.
  • a decoder comprising the bidirectional inter prediction apparatus and the reconstruction module in the second aspect, wherein the reconstruction module is configured to generate a prediction image according to the bidirectional inter prediction apparatus The pixels of the block determine the reconstructed pixel value of the current image block.
  • a computer readable storage medium stores an instruction, when the instruction is run on the terminal described in the third aspect, causing the terminal to perform the foregoing A bidirectional inter prediction method as described on the one hand and any of its possible implementations.
  • a ninth aspect a computer program product comprising instructions for causing the terminal to perform the first aspect as described above and any one of possible implementations thereof when the computer program product is run on the terminal described in the third aspect above
  • the bidirectional inter prediction method described in the mode is described in the mode.
  • the difference value may be based on a sum of absolute sum values (Sum of Absolute Differences, SAD) and absolute values of error transform coefficients (Sum of Absolute Transformed Difference, SATD) or Mean-Removal Sum of Absolute Differences (MR-SAD) is calculated.
  • SAD Sum of Absolute Differences
  • SATD absolute values of error transform coefficients
  • MR-SAD Mean-Removal Sum of Absolute Differences
  • the name of the above-mentioned bidirectional inter prediction apparatus is not limited to the device or the function module itself. In actual implementation, these devices or function modules may appear under other names. As long as the functions of the respective devices or functional modules are similar to the present application, they are within the scope of the claims and their equivalents.
  • the second aspect to the ninth aspect of the present application may refer to the detailed description in the first aspect and various implementations thereof; and the second aspect to the ninth aspect, and each of
  • FIG. 1 is a schematic structural diagram of a CTU in an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a template of an image block in an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a video transmission system according to an embodiment of the present disclosure.
  • FIG. 4A is a schematic structural diagram of a video codec apparatus according to an embodiment of the present disclosure.
  • 4B is a schematic structural diagram of a video codec system according to an embodiment of the present application.
  • FIG. 5 is a schematic flowchart 1 of a bidirectional inter-frame prediction method according to an embodiment of the present disclosure
  • FIG. 6A is a schematic structural diagram 1 of a current template, a first reference template, and a second reference template according to an embodiment of the present application;
  • FIG. 6B is a schematic structural diagram 2 of a current template, a first reference template, and a second reference template according to an embodiment of the present disclosure
  • FIG. 7 is a second schematic flowchart of a bidirectional inter-frame prediction method according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic structural diagram of a current image block, a search template, and a predicted image block according to an embodiment of the present application;
  • FIG. 9 is a schematic flowchart 3 of a bidirectional inter-frame prediction method according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of a bidirectional inter prediction apparatus according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a bidirectional inter prediction apparatus according to an embodiment of the present application.
  • the words “exemplary” or “such as” are used to mean an example, illustration, or illustration. Any embodiment or design described as “exemplary” or “for example” in the embodiments of the present application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the words “exemplary” or “such as” is intended to present the concepts in a particular manner.
  • Coding Tree Unit An image consists of multiple CTUs.
  • a CTU usually corresponds to a square image area.
  • image 10 consists of multiple CTUs (including CTU A and CTU B). , CTU C, etc.).
  • the coding information corresponding to a certain CTU includes luminance values and/or chrominance values of pixels in a square image region corresponding to the CTU.
  • the coding information corresponding to a certain CTU may further include syntax elements indicating how to divide the CTU into at least one Coding Unit (CU), and a method of decoding each CU to obtain a reconstructed image.
  • CU Coding Unit
  • An image area corresponding to one CTU may include 64 ⁇ 64, 128 ⁇ 128, or 256 ⁇ 256 pixels.
  • a CTU of 64x64 pixels contains a rectangular pixel lattice of 64 columns of 64 pixels per column, each pixel containing a luminance component and/or a chrominance component.
  • the CTU can also correspond to a rectangular image area or an image area of other shapes.
  • the image area corresponding to one CTU may also be an image area in which the number of pixels in the horizontal direction is different from the number of pixels in the vertical direction, for example, including 64 ⁇ 128 pixels.
  • CU generally corresponds to a rectangular area of A ⁇ B, A is the width of the rectangle, and B is the height of the rectangle.
  • the width referred to in the embodiment of the present application refers to the two-dimensional Cartesian coordinate system XoY shown in FIG. 1 .
  • the length in the X-axis direction (horizontal direction), the height refers to the length in the Y-axis direction (vertical direction) in the two-dimensional Cartesian coordinate system XoY shown in FIG.
  • the values of A and B may be the same or different.
  • the values of A and B are usually an integer power of 2, for example: 256, 128, 64, 32, 16, 8, or 4.
  • a CU can obtain a reconstructed image block of a rectangular area of A ⁇ B by decoding processing, and the decoding process usually includes processing such as prediction, dequantization, inverse transform, etc., to generate a predicted image and a residual image.
  • the reconstructed image block is obtained by superimposing the predicted image and the residual image.
  • a plurality of reconstructed image blocks can be used to obtain a final reconstructed image.
  • Slice consists of a series of CTUs arranged in raster scan order. An image can be decomposed into one or several stripes, each of which includes a set of CTUs.
  • Inter-prediction refers to the prediction performed by the correlation between the current frame and its reference frame in units of coded image blocks/decoded image blocks.
  • One or more reference frames may exist in the current frame.
  • the predicted image block of the current image block is generated according to the pixels in the reference frame of the current image block.
  • the predicted image block of the current image block may be generated from only one reference image block, or the predicted image block of the current image block may be generated according to the two reference image blocks.
  • the above-described predicted image block for generating a current image block from one reference image block is referred to as unidirectional prediction
  • the above-described predicted image block for generating a current image block from two reference image blocks is referred to as bidirectional inter prediction.
  • Two reference picture blocks in bi-directional inter prediction may come from the same reference frame or different reference frames.
  • the bi-directional inter-prediction may refer to the correlation between the current video frame and the video frame previously encoded and played before it, and the current video frame and the video encoded before and after the video frame. Inter-prediction by correlation between frames.
  • Forward inter-prediction refers to inter-prediction using the correlation between the current video frame and a video frame that was previously encoded and played before it.
  • Backward inter prediction refers to inter prediction using the correlation between the current video frame and a video frame that was previously encoded and played after it.
  • the forward inter prediction corresponds to the forward reference frame list L0
  • the backward inter prediction corresponds to the backward reference frame list L1.
  • the number of reference frames included in the two reference frame lists may be the same or different.
  • Image encoding The process of compressing a sequence of images into a stream of code.
  • Image decoding The process of restoring a code stream into a reconstructed image according to specific grammar rules and processing methods.
  • a video sequence consists of a series of pictures, the image being divided into at least one strip, each strip being divided into image blocks.
  • the video encoding/decoding is performed in units of image blocks, and encoding/decoding processing can be performed from left to right, top to bottom, and line by line from the upper left corner position of the image.
  • the image block may be a macro block (MB) in the video codec standard H.264, or may be a CU in the High Efficiency Video Coding (HEVC) standard, which is not used in this embodiment of the present application. Specifically limited.
  • an image block in which encoding/decoding processing is being performed is referred to as a current image block
  • an image in which the current image block is located is referred to as a current frame.
  • the current frame may be a unidirectional prediction frame (P frame) or a bidirectional prediction frame (B frame).
  • the current frame has a list of reference frames.
  • the current frame has two reference frame lists, which are generally referred to as L0 and L1, respectively.
  • Each reference frame list contains at least one reconstructed frame that serves as a reference frame for the current frame.
  • the reference frame is used to provide reference pixels for inter prediction of the current frame.
  • the image blocks adjacent to the current image block may have completed the encoding/decoding process, resulting in reconstructed images, which are referred to as reconstructed image blocks.
  • the information of reconstructing the coding mode of the image block, reconstructing pixels, and the like is available.
  • a frame that has completed encoding/decoding processing before encoding/decoding of the current frame is referred to as a reconstructed frame.
  • Template of an image block includes a pixel region that is adjacent to the image block and has been reconstructed. There may be no overlapping area between the image block and the template of the image block.
  • the template of the image block includes at least one row of reconstructed pixels adjacent to the image block and located in a certain direction (at least one of the top, left, and right sides) of the image block.
  • the shape of the template may be L-shaped or other shapes, which is not specifically limited in this application.
  • the adjacent L-shaped pixel area 20' around the image block 20 is a template of the image block 20
  • the pixel area 21' located on the left side of the image block 21 and adjacent to the image block 21 is the image block 21.
  • the template, the pixel area 22' located on the left side of the image block 22, up and down, and adjacent to the image block 22 is a template of the image block 22.
  • Motion Vector indicates the relative displacement between the current image block and the reference image block.
  • video images are mainly encoded by motion compensated interframe prediction techniques.
  • the encoding end encodes the current image block in the current frame
  • first, one or more reference frames are arbitrarily selected from the encoded frame of the video image, and the predicted image block corresponding to the current image block is obtained from the reference frame. And calculating a residual value between the predicted image block and the current image block, and performing quantization quantization on the residual value;
  • the decoding end first acquires the predicted image corresponding to the current image block. And then obtaining a residual value of the predicted image block from the current image block from the received code stream, and reconstructing the current image block according to the residual value and the prediction block decoding.
  • the temporal correlation between the current frame and other frames in the video is not only reflected in the temporal correlation between the current frame and the frame encoded before it, but also in the temporal correlation between the current frame and the frame encoded after it. . Based on this, bidirectional inter prediction can be considered when performing video coding to obtain a better coding effect.
  • bidirectional inter prediction In the process of bidirectional inter prediction, two reference image blocks are weighted according to respective weight values to generate a predicted image block of the current image block.
  • bidirectional inter prediction is to weight two reference image blocks according to the weight value of the strip level to generate a prediction image block, or use a default weight value (1/2, 1/2).
  • the two reference image blocks are uniformly weighted to generate a predicted image block.
  • the predicted image block generated using the existing method is less similar to the current image block.
  • the embodiment of the present application provides a bidirectional inter prediction method, after determining the first reference image block and the second reference image block, acquiring a first reference template (the first reference template includes the first reference image block) An adjacent and reconstructed pixel region), a second reference template (the second reference template includes a pixel region adjacent to the second reference image block and reconstructed), and a current template (the current template includes adjacent to the current image block) And the reconstructed pixel area), then calculating a first difference value between the current template and the first reference template, and a second difference value between the current template and the second reference template, and determining to represent the first a difference between the difference and the difference between the second difference value, and further determining, according to the preset range of the difference, the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, so that Weighting a pixel of the first reference image block and a pixel of the second reference image block by weighting a weight value corresponding to the reference image block and a
  • the positional relationship between the current template and the current image block, the positional relationship between the first reference template and the first reference image block, and the positional relationship between the second reference template and the second reference image block are the same.
  • the gap belongs to different preset ranges
  • the weight values corresponding to the first reference image block are different
  • the weight values corresponding to the second reference image block are different.
  • the bidirectional inter prediction method provided by the present application determines the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, that is, the weight value of the block level is determined, compared to the current
  • the bidirectional inter-frame prediction method provided by the embodiment of the present application effectively improves the similarity between the current image block and the predicted image block.
  • the bidirectional inter prediction method provided by the present application may be performed by a bidirectional inter prediction apparatus, a video codec apparatus, a video codec, and other devices having a video codec function.
  • the bi-directional inter-frame prediction method may be performed in the encoding process or in the decoding process, which is not specifically limited in this embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a video transmission system according to an embodiment of the present application.
  • the video transmission system includes an acquisition module 3001, an encoding module 3002, a transmitting module 3003, a network 3004, a receiving module 3005, a decoding module 3006, and a rendering module 3007.
  • each module in the video transmission system is as follows:
  • the acquisition module 3001 includes a camera or a camera group for collecting video images, and performing pre-encoding processing on the captured video images to convert the optical signals into digitized video sequences.
  • the encoding module 3002 is configured to encode the video sequence to obtain a code stream.
  • the sending module 3003 is configured to send the coded code stream.
  • the receiving module 3005 is configured to receive the code stream sent by the sending module 3003.
  • the network 3004 is configured to transmit the code stream sent by the sending module 3003 to the receiving module 3005.
  • the decoding module 3006 is configured to decode the code stream received by the receiving module 3005 to reconstruct a video sequence.
  • the rendering module 3007 is configured to render the reconstructed video sequence decoded by the decoding module 3006 to improve the display effect of the video.
  • the encoding module 3001 and the decoding module 3006 in the video transmission system shown in FIG. 3 can perform the bidirectional inter prediction method in the embodiment of the present application.
  • the bidirectional inter prediction method provided by the embodiment of the present application can be performed by a video codec device.
  • the bi-directional inter-frame prediction method provided by the embodiment of the present application is also applicable to a codec system composed of a video codec device.
  • the codec device and the codec system composed of the codec device will be described in detail below with reference to FIGS. 4A and 4B.
  • FIG. 4A is a schematic diagram of a video codec apparatus according to an embodiment of the present application.
  • the video codec device 40 may be a device dedicated to encoding and/or decoding a video image, or may be an electronic device having a video codec function. Further, the video codec device 40 may be It is a mobile terminal or user equipment of a wireless communication system.
  • Video codec device 40 may include the following modules or units: controller 401, codec 402, radio interface 403, antenna 404, smart card 405, card reader 406, keypad 407, memory 408, infrared port 409, display 410. Moreover, in addition to the modules and units shown in FIG. 4A, video codec device 40 may also include a microphone or any suitable audio input module, which may be a digital or analog signal input. Correspondingly, the video codec device 40 may further comprise an audio output module, which may be a headset, a speaker or an analog audio or digital audio output connection. Video codec device 40 may also include a battery, which may be a solar cell, a fuel cell, or the like.
  • the video codec device 40 may also include an infrared port for short-range line-of-sight communication with other devices, and the video codec device 40 may also communicate with other devices, including Bluetooth wireless connections, including any suitable short-range communication method. , USB / Firewire wired connection.
  • the memory 408 can store data in the form of data and audio of an image, and can also store instructions for execution on the controller 401.
  • Video codec 40 may implement encoding and decoding of audio and/or video data or assisted encoding and auxiliary decoding of audio and/or video data under the control of controller 401.
  • the smart card 405 and the card reader 406 can provide user information as well as network authentication and authentication information for authorized users.
  • the specific implementation form of the smart card 405 and the card reader 406 may be a Universal Integrated Circuit Card (UICC) and a UICC reader.
  • UICC Universal Integrated Circuit Card
  • the radio interface 403 can generate a wireless communication signal, which can be a communication signal generated during a cellular communication network, a wireless communication system, or a wireless local area network communication.
  • the antenna 404 is used to transmit radio frequency signals generated in the radio interface circuit 403 to other devices (the number of devices may be one or more), and may also be used from other devices (the number of devices may be one or more Receive RF signals.
  • video codec device 40 may receive video image data to be processed transmitted by another device prior to transmission and/or storage. In still other embodiments of the present application, video codec device 40 may receive images over a wireless or wired connection and encode/decode the received images.
  • FIG. 4B is a schematic block diagram of a video codec system 50 in the embodiment of the present application.
  • the video codec system 50 includes a source device 501 and a destination device 502.
  • the source device 501 generates encoded video data
  • the source device 501 may also be referred to as a video encoding device or a video encoding device.
  • the destination device 502 may decode the encoded video data generated by the source device 501, and the destination device 502 also It may be referred to as a video decoding device or a video decoding device.
  • the specific implementation form of the source device 501 and the destination device 502 may be any one of the following devices: desktop computer, mobile computing device, notebook (eg, laptop) computer, tablet computer, set top box, smart phone, handset, television , camera, display device, digital media player, video game console, on-board computer, or other similar device.
  • desktop computer mobile computing device
  • notebook (eg, laptop) computer tablet computer
  • set top box smart phone
  • handset television
  • camera camera
  • display device digital media player
  • video game console on-board computer, or other similar device.
  • the destination device 502 can receive the video data encoded by the source device 501 via the channel 503.
  • Channel 503 can include one or more media and/or devices capable of moving encoded video data from source device 501 to destination device 502.
  • channel 503 can include one or more communication media that enable source device 501 to transmit encoded video data directly to destination device 502 in real time, in which case source device 501 can be based on communication standards (eg, The wireless communication protocol is used to modulate the encoded video data, and the modulated video data can be transmitted to the destination device 502.
  • the one or more communication media may include wireless and/or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines.
  • RF radio frequency
  • the one or more communication media described above may form part of a packet-based network (eg, a local area network, a wide area network, or a global network (eg, the Internet)).
  • the one or more communication media described above may include routers, switches, base stations, or other devices that enable communication from source device 501 to destination device 502.
  • channel 503 can include a storage medium that stores encoded video data generated by source device 501.
  • destination device 502 can access the storage medium via disk access or card access.
  • the storage medium may include a variety of locally accessible data storage media, such as Blu-ray Disc, High Density Digital Video Disc (DVD), Compact Disc Read-Only Memory (CD-ROM), flash memory. Or other suitable digital storage medium for storing encoded video data.
  • channel 503 can include a file server or another intermediate storage device that stores encoded video data generated by source device 501.
  • destination device 502 can access encoded video data stored at a file server or other intermediate storage device via streaming or download.
  • the file server may be a server type capable of storing encoded video data and transmitting the encoded video data to the destination device 502.
  • the file server may include a World Wide Web (Web) server (for example, for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk. driver.
  • Web World Wide Web
  • FTP File Transfer Protocol
  • NAS Network Attached Storage
  • the destination device 502 can access the encoded video data via a standard data connection (e.g., an internet connection).
  • the instance type of the data connection includes a wireless channel, a wired connection (e.g., a cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on the file server.
  • the transmission of the encoded video data from the file server may be streaming, downloading, or a combination of both.
  • the bidirectional inter prediction method of the present application is not limited to a wireless application scenario.
  • the bidirectional inter prediction method of the present application can be applied to video codec supporting multiple multimedia applications such as the following applications: aerial television broadcasting, cable television transmission, Satellite television transmission, streaming video transmission (e.g., via the Internet), encoding of video data stored on a data storage medium, decoding of video data stored on a data storage medium, or other application.
  • video codec system 50 can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.
  • the source device 501 includes a video source 5011, a video encoder 5012, and an output interface 5013.
  • output interface 5013 can include a modulator/demodulator (modem) and/or a transmitter.
  • Video source 5011 may include a video capture device (eg, a video camera), a video archive containing previously captured video data, a video input interface to receive video data from a video content provider, and/or a computer for generating video data.
  • Video encoder 5012 can encode video data from video source 5011.
  • source device 501 transmits the encoded video data directly to destination device 502 via output interface 5013.
  • the encoded video data may also be stored on a storage medium or file server for later access by the destination device 502 for decoding and/or playback.
  • destination device 502 includes an input interface 5023, a video decoder 5022, and a display device 5021.
  • input interface 5023 includes a receiver and/or a modem.
  • Input interface 5023 can receive the encoded video data via channel 503.
  • Display device 5021 can be integrated with destination device 502 or can be external to destination device 502. Generally, the display device 5021 displays the decoded video data.
  • Display device 5021 can include a variety of display devices, such as liquid crystal displays, plasma displays, organic light emitting diode displays, or other types of display devices.
  • the video encoder 5012 and the video decoder 5022 can operate according to a video compression standard (eg, the high efficiency video codec H.265 standard), and can follow a High Efficiency Video Coding (HEVC) test model (HM). .
  • a video compression standard eg, the high efficiency video codec H.265 standard
  • HEVC High Efficiency Video Coding test model
  • the bidirectional inter prediction method provided by the present application is described in detail below in conjunction with the video transmission system shown in FIG. 3, the video codec device shown in FIG. 4A, and the codec system composed of the video codec device shown in FIG. 4B.
  • the bidirectional inter prediction method provided by the embodiment of the present application may be performed by a bidirectional inter prediction apparatus, may also be performed by a video codec apparatus, may also be performed by a video codec, and may also be executed by other devices having a video codec function.
  • the embodiment of the present application does not specifically limit this.
  • FIG. 5 is a schematic flowchart diagram of a bidirectional inter-frame prediction method according to an embodiment of the present application.
  • the bi-directional inter-frame prediction method shown in FIG. 5 can occur both in the encoding process and in the decoding process. More specifically, the bi-directional inter-frame prediction method shown in FIG. 5 can occur in the inter-frame prediction process at the time of encoding and decoding.
  • bidirectional inter-frame prediction apparatus will be described as an example of the execution main body in FIG.
  • the bidirectional inter prediction method includes:
  • the bidirectional inter prediction apparatus determines the first reference image block and the second reference image block according to the bidirectional motion information of the current image block.
  • the bidirectional motion information of the current image block includes a first reference frame index for indicating the first reference frame, a second reference frame index for indicating the second reference frame, a first motion vector, and a second motion vector.
  • the bidirectional inter prediction apparatus determines the first reference image block and the second reference image block by: the bidirectional inter prediction apparatus acquires the first reference frame index, the second reference frame index, the first motion vector, and a second motion vector; the bidirectional inter prediction apparatus acquires a first reference frame from the first reference frame list according to the first reference frame index, and acquires the second reference frame from the second reference frame list according to the second reference frame index The bidirectional inter prediction apparatus determines the first reference image block according to the first motion vector and the first reference frame, and determines the second reference image block according to the second motion vector and the second reference frame.
  • the first reference frame and the second reference frame may be the same reference frame, or may be different reference frames, which is not specifically limited in this embodiment of the present application.
  • the following content of the embodiment of the present application is described by taking the first reference frame and the second reference frame as different reference frames as an example.
  • the first reference frame index and the second reference frame index in the embodiment of the present application may be identified in the code stream.
  • syntax elements ref_idx_l0 and ref_idx_l1 represent reference frame indices in the forward reference frame list L0 and the backward reference frame list L1, respectively.
  • the bidirectional inter prediction apparatus acquires the first reference frame index and the second reference frame index from the code stream.
  • the bidirectional inter prediction device may also take a reference frame index corresponding to the forward motion vector as a first reference index from a bidirectional motion vector predictor of the current image block, and take a reference frame index corresponding to the backward motion vector. As the second reference frame index.
  • the bidirectional motion vector prediction value is one motion vector prediction value in a Motion Vector Predictor candidate list (MVP candidate list) of the current image block.
  • the motion vector predictor candidate list of the current image block may include a one-way motion vector or a two-way motion vector.
  • the bidirectional inter prediction apparatus may further use a reference frame index of a forward motion vector predictor of the current image block as a first reference index, and use a reference frame index of a backward motion vector predictor of the current image block as a reference frame.
  • a reference index may be used.
  • the forward motion vector prediction value is one motion vector prediction value in the forward motion vector predictor candidate list of the current image block
  • the backward motion vector prediction value is a backward motion vector predictor candidate list of the current image block.
  • a motion vector predictor in The forward motion vector predictor candidate list of the current image block includes a unidirectional motion vector
  • the backward motion vector predictor candidate list of the current image block includes a unidirectional motion vector.
  • the bidirectional inter-frame prediction apparatus in the embodiment of the present application may also obtain the first reference frame index and the second reference frame index in other manners, which is not specifically limited in this embodiment of the present application.
  • the first motion vector and the second motion vector in the embodiment of the present application are both unidirectional motion vectors.
  • the first motion vector and the second motion vector may be obtained by adding a Motion Vector Predictor (MVP) and a Motion Vector Difference (MVD), or may be determined only based on the motion vector predictor. It can be obtained by decoding the motion vector search.
  • MVP Motion Vector Predictor
  • MVD Motion Vector Difference
  • the manner in which the first motion vector and the second motion vector are determined in the embodiment of the present application is not specifically limited.
  • the bidirectional inter prediction apparatus may further determine the first reference image block in the first reference frame according to the search template, and determine the second reference image block in the second reference frame according to the search template. This method is described in detail in the embodiment described in Figures 7 and 9.
  • the bidirectional inter prediction device acquires a current template, a first reference template, and a second reference template.
  • the current template includes a pixel region adjacent to the current image block and reconstructed
  • the first reference template includes a pixel region adjacent to the first reference image block and reconstructed
  • the second reference template includes adjacent to the second reference image block And the reconstructed pixel area.
  • the positional relationship between the current template and the current image block “the positional relationship between the first reference template and the first reference image block”, and “the positional relationship between the second reference template and the second reference image block” are the same.
  • the template of a certain image block may include adjacent, reconstructed pixel regions of at least one of the upper side, the left side, and the right side of the image block.
  • T0 the first reference template
  • T1 the second reference template
  • Tc the current template If the current template Tc is adjacent to the current image block, it is located on the left side of the current image block.
  • the first reference template T0 includes two columns of pixels adjacent to the first reference image block and located on the left side of the first reference image block, and Two rows of pixels reconstructed on the upper side of the first reference image block
  • the second reference template T1 includes two columns of pixels adjacent to the second reference image block, located on the left side of the second reference image block, and located in the second reference image Two rows of pixels that have been reconstructed on the top side of the block.
  • T0 be the first reference template
  • T1 be the second reference template
  • Tc be the current template. If the current template Tc is adjacent to the current image block, it is located on the left side of the current image block.
  • the reconstructed column of pixels, the first reference template T0 includes a column of pixels adjacent to the first reference image block that is reconstructed on the left side of the first reference image block, and the second reference template T1 includes adjacent to the second reference image block. A column of pixels that have been reconstructed on the left side of the second reference image block.
  • the bidirectional inter prediction apparatus may acquire the current template after determining the first reference image block and the second reference image block, and may also acquire the current template before determining the first reference image block and the second reference image block. This is not specifically limited.
  • the template of a certain image block may include adjacent, reconstructed pixel regions of at least one of the upper side, the left side, and the right side of the image block.
  • the bidirectional inter prediction apparatus may determine the template of the image block according to an area that has been reconstructed around a certain image block (the current image block, the first reference image block, or the second reference image block).
  • the current template includes an upper adjacent area of the current image block; if both the upper side and the left adjacent area of the current image block have been reconstructed, The current template includes an upper adjacent area and a left adjacent area of the current image block; if both the upper side and the right adjacent area of the current image block have been reconstructed, the current template includes an upper adjacent area of the current image block and Adjacent area on the right.
  • the bidirectional inter prediction apparatus may further perform the similarity between the motion vector of the adjacent region of the certain image block (the current image block, the first reference image block, or the second reference image block) and the motion vector of the image block. , determine the template for the image block.
  • the bidirectional inter prediction apparatus determines whether the forward motion vector of at least one of the upper adjacent area, the left adjacent area, and the right adjacent area of the current image block is forward motion vector of the current image block. Similarly, and determining that the forward motion vector is similar to the current template; or the bidirectional inter prediction device determines the at least one of the upper adjacent region, the left adjacent region, and the right adjacent region of the current image block Whether the motion vector is similar to the backward motion vector of the current image block and determines that the backward motion vector is similar to the current template.
  • the judging method that the bidirectional inter prediction apparatus determines that the two motion vectors are similar may be: if two motion vectors point to the same reference frame and the sum of the absolute values of the horizontal component difference and the vertical component difference between the two is smaller than If the threshold is preset (for example, 4, 8 or 12 pixels), then the two motion vectors are similar, otherwise they are not similar.
  • the bi-directional inter-frame prediction apparatus in the embodiment of the present application may also obtain a template of a certain image block in other manners, which is not specifically limited in this embodiment of the present application.
  • the bidirectional inter prediction device may directly directly convert the first reference image block and the second after performing S501.
  • the weight values corresponding to the reference image blocks are determined to be 1/2 and 1/2.
  • the bidirectional inter prediction device calculates a first difference value between the current template and the first reference template, and a second difference value between the current template and the second reference template.
  • the current template is Tc
  • the first reference template is T0
  • the second reference template is T1
  • the first difference value is S0
  • the second difference value is S1.
  • the bidirectional inter prediction apparatus may determine the difference value according to the SAD, that is, the bidirectional inter prediction apparatus determines S0 according to the SAD of the Tc and T0 corresponding pixels, and determines S1 according to the SAD of the corresponding pixel of Tc and T1.
  • the bidirectional inter prediction apparatus determines the difference value based on the SAD
  • the bidirectional inter prediction apparatus calculates S0 based on the following SAD calculation formula (1), and calculates S1 according to the following SAD calculation formula (2).
  • f c (x, y) is the luminance component value of a pixel ((pixel, (x, y)) whose position is (x, y) in the current image block, pixel (x, y) is offset from the position (x, y) of the pixel (0, 0) in the upper left corner of the current image block, where x is the horizontal offset and y is the vertical offset; N1 is the current The length of the image block in the horizontal direction, N2 is the length of the current image block in the vertical direction; f 0 (x-dx0, y-dy0) is the first pixel (x, y) obtained from the motion vector (dx0, dy0) The luminance component value of the corresponding position in the reference frame; f 1 (x-dx1, y-dy1) is the luminance component value of the corresponding position in the second reference frame obtained by the pixel (x, y) according to the motion vector (dx1, dy
  • the difference values in the above formula (1) and formula (2) are SADs of the luminance component values.
  • the difference value in the embodiment of the present application may also be the sum of the SAD of the luminance component value and the SAD of the chrominance component value.
  • the bidirectional inter prediction apparatus may further determine the difference value according to the SATD, that is, the bidirectional inter prediction apparatus determines S0 according to the SATD of the corresponding pixel of Tc and T0, and determines S1 according to the SATD of the corresponding pixel of Tc and T1.
  • the bidirectional inter prediction apparatus determines the difference value based on the SATD
  • the bidirectional inter prediction apparatus calculates S0 based on the SATD calculation formula (3) described below, and calculates S1 according to the SATD calculation formula (4) described below.
  • the bidirectional inter prediction apparatus may further determine a difference value according to the MR-SAD, that is, the bidirectional inter prediction apparatus determines S0 according to the MR-SAD of the Tc and T0 corresponding pixels, and determines the S1 according to the MR-SAD of the corresponding pixel of Tc and T1. .
  • the bidirectional inter prediction apparatus determines the difference value based on the MR-SAD
  • the bidirectional inter prediction apparatus calculates S0 according to the following MR-SAD calculation formula (5), and calculates the formula (6) according to the following MR-SAD. Calculate S1.
  • DeltaC0 is the difference between the mean values of the brightness of Tc and T0
  • DeltaC1 is the difference between the mean values of the brightness of Tc and T1, ie
  • the bi-directional inter-frame prediction apparatus in the embodiment of the present application can also determine the difference value according to other criteria, which will not be exemplified herein.
  • the bidirectional inter prediction apparatus determines a difference used to represent a difference degree of the first difference value and the second difference value, and determines a weight value corresponding to the first reference image block and the second reference image according to the preset range of the gap attribution. The weight value corresponding to the block.
  • the prediction device may uniformly weight the two reference image blocks, that is, the weight value corresponding to the first reference image block is equal to the weight value corresponding to the second reference image block.
  • the difference between the first difference value and the second difference value is large, the first difference image value is far less than the second difference value as an example, and the first reference image block is more similar to the current image block than the first image block.
  • the bidirectional inter prediction device assigns a larger weight value to the first reference image block and assigns a smaller weight value to the second reference image block, thus enabling The predicted image block is statistically closer to the current image block.
  • the sum of the weight value corresponding to the first reference image block and the second reference image block is equal to one.
  • the bidirectional inter prediction device first determines a difference for characterizing a degree of difference between the first difference value and the second difference value.
  • the bidirectional inter prediction apparatus determines a difference D for characterizing a degree of difference between the difference values of the two reference image blocks and the current image block according to the following formula (7) or the following formula (8), and according to the
  • the gap D is a weight value assigned to the first reference image block and the second reference image block.
  • n is the first preset constant
  • the greater the difference between the two difference values ie, the closer the value of D in the above formula (7) and formula (8) is closer to 0 or positive infinity), the more uneven the weight distribution, that is, the difference in weight value corresponding to the two reference image blocks.
  • the smaller the difference between the two difference values that is, the closer the value of D in the above formula (7) and formula (8) is to 1), the more uniform the weight distribution, that is, the smaller the weight value corresponding to the two reference image blocks.
  • the bidirectional inter prediction apparatus determines a difference D for characterizing a degree of difference between the difference values of the two reference image blocks and the current image block according to the following formula (9) or the following formula (10), and according to The gap D is a weight value assigned to the first reference image block and the second reference image block.
  • the greater the difference between the two difference values that is, the closer the value of D in the above formula (9) and formula (10) is to the near infinity or the positive infinity), the more uneven the weight distribution, that is, the weight value corresponding to the two reference image blocks.
  • the difference is greater.
  • the smaller the difference between the two difference values that is, the closer the value of D in the above formula (9) and formula (10) is closer to 0), the more uniform the weight distribution, that is, the smaller the weight value corresponding to the two reference image blocks.
  • a simple weight distribution method is: according to the gap D, a set of weight values is selected as a weight value corresponding to two reference image blocks in a preset plurality of weight values.
  • the bidirectional inter prediction device After determining the gap D, the bidirectional inter prediction device determines a weight value corresponding to the first reference image block and a weight value corresponding to the second reference image block according to the preset range to which the gap D belongs.
  • the bidirectional inter prediction device may determine, according to the preset range of the gap D and the preset correspondence, the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block; And determining, by the preset formula, a weight value corresponding to the first reference image block and a weight value corresponding to the second reference image block.
  • the "two-way inter prediction apparatus determines the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block according to D and the preset correspondence relationship".
  • the bidirectional inter prediction apparatus searches for a weight value offset corresponding to the preset range to which the gap D belongs from the preset correspondence, such that the bidirectional inter prediction apparatus determines the offset of the weight value and 1/1
  • the sum of 2 is a weight value corresponding to the first reference image block, and the difference between the weight values corresponding to the first reference image block is determined to be a weight value corresponding to the second reference image block.
  • the preset correspondence includes a plurality of preset gap ranges and a weight value offset corresponding to each preset gap range, and the preset range is one of the plurality of preset gap ranges.
  • ⁇ w is a positive value ( As 1/4 of Table 1, correspondingly, the bidirectional inter prediction apparatus determines that the weight value ⁇ w0, w1 ⁇ is ⁇ 3/4, 1/4 ⁇ .
  • the bidirectional inter prediction device determines the weight value ⁇ w0, w1 ⁇ is ⁇ 1/2, 1/2 ⁇ .
  • ⁇ w is a negative value (such as -1/4 in Table 1), and correspondingly, the bidirectional inter prediction apparatus determines the weight value ⁇ w0, w1 ⁇ For ⁇ 1/4, 3/4 ⁇ .
  • the value of SmThd1 may be 0.5, and the value of thred1 may be 2.
  • the degree of difference between the two difference values is calculated using the above formula (8), the precision of the weight value is 1/8, the preset correspondence is as shown in Table 2, and w0 is the first reference image block.
  • w1 is a weight value corresponding to the second reference image block
  • the uniform weighted weight value is Wa
  • the weight value corresponding to the second reference image block Is w1 Wa- ⁇ w.
  • SmThd3, SmThd2, SmThd1, thred1, thred2, and thred3 in Table 2 are used to indicate different thresholds, where the value of thred1 can be 1.5, the value of thred2 can be 2, the value of thred3 can be 3, and the value of SmThd1 can be 2/3, the value of SmThd2 can be 0.5, and the value of SmThd3 can be 1/3.
  • the degree of difference between the two difference values is calculated using the above formula (8), the precision of the weight value is 1/16, the preset correspondence is as shown in Table 3, and w0 is the first reference image block.
  • w1 is a weight value corresponding to the second reference image block
  • the uniform weighted weight value is Wa
  • the weight value corresponding to the second reference image block Is w1 Wa- ⁇ w.
  • SmThd7, SmThd6, SmThd5, SmThd4, SmThd3, SmThd2, SmThd1, thred1, thred2, thred3, thred4, thred5, thred6, and thred7 in Table 3 are used to represent different thresholds, where the value of thred1 can be 1.25, and the value of thred2 can be For 1.5, the value of thred3 can be 1.75, the value of thred4 can be 2, the value of thred5 can be 2.25, the value of thred6 can be 2.5, the value of thred7 can be 2.75, and the value of thmd[i] is reciprocal of thred[i]. i is 1, 2, ..., 7.
  • the threshold in the preset correspondence (such as Table 1, Table 2, and Table 3 above) may be encoded by the index Columbus, included in the code stream of the sequence header, or the threshold in the preset correspondence is The threshold is preset.
  • the above example can also avoid the division by the following implementation, and reduce the complexity: take the above Table 1 as an example, if (S1 n + k) > thred1 ⁇ (S0 n +k), then ⁇ w is 1/4, the weight value ⁇ w0, w1 ⁇ is ⁇ 3/4, 1/4 ⁇ ; if (S1 n + k) ⁇ SmThd1 ⁇ (S0 n + k), then ⁇ w is - 1/4, the weight value ⁇ w0, w1 ⁇ is ⁇ 1/4, 3/4 ⁇ ; otherwise, ⁇ w is 0, and the weight value ⁇ w0, w1 ⁇ is ⁇ 1/2, 1/2 ⁇ .
  • the "two-way inter prediction apparatus determines the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block according to D and the preset formula".
  • the bidirectional inter prediction device determines, according to a preset formula, a weight value offset corresponding to a preset range to which the gap D belongs, and determines that the weight value corresponding to the first reference image block is equal to 1/2 and the weight value.
  • the sum of the offsets, the weight value corresponding to the second reference image block is 1 minus the weight value corresponding to the first reference image block.
  • the amount, ⁇ w is usually a non-negative number.
  • the bidirectional inter prediction apparatus determines the weight value offset amount ⁇ w according to the following formula (11) or formula (12).
  • floor( ⁇ ) is a downward rounding function
  • P is the precision of the weighting value, such as 1/4, 1/8 or 1/16, etc.
  • o is the fourth preset Constant
  • s represents the step size of the linear relationship
  • its value is the preset constant
  • q represents the step size of the logarithmic relationship
  • its value is the fifth preset constant
  • i is the sixth preset constant
  • the values of related parameters (such as P, o, s, etc.) in the preset formula (such as formula (11), formula (12)) can be encoded by the index Columbus, and the code stream included in the sequence header is used.
  • the values of the relevant parameters in the middle or preset formula are all preset values.
  • the bidirectional inter prediction apparatus determines that the weight value ⁇ w0, w1 ⁇ is ⁇ 1/2, 1/2 ⁇ .
  • min( ⁇ ) is a smaller value operation
  • NumP is the number of pixels included in the current image block, that is, the width and current of the current image block. The high product of the image block.
  • the bidirectional inter prediction apparatus determines that the weight value ⁇ w0, w1 ⁇ is ⁇ 1/2, 1/2 ⁇ .
  • the bidirectional inter prediction apparatus weights the first reference image block and the second reference image block according to the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, to obtain a predicted image block.
  • the bidirectional inter prediction device After determining the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, the bidirectional inter prediction device determines, according to the weight value corresponding to the first reference image block and the weight value corresponding to the second reference image block, Pixels of the first reference image block and the second reference image block are weighted to obtain pixels of the predicted image block of the current image block.
  • the bidirectional inter prediction apparatus may determine the predicted image block P C according to the following formula (13).
  • w0 is a weight value corresponding to the first reference image block
  • w1 is a weight value corresponding to the second reference image block
  • f 0 is a current image block obtained according to the first motion vector (dx0, dy0).
  • f 1 is a second reference image block of the corresponding position in the second reference frame obtained by the current image block according to the second motion vector (dx1, dy1).
  • the pixel involved in the embodiment of the present application may only include a luminance component value, and may also include a luminance component value and a chrominance component value.
  • the first motion vector (dx0, dy0) and the second motion vector (dx1, dy1) are sub-pixel precision motion vectors
  • the first reference image block f 0 and the second reference image block f 1 are The integer pixel interpolation of the spatial position.
  • the interpolation method herein may be a sub-pixel interpolation method in the H.265 standard, or may be a linear interpolation method, or may be another interpolation method, which is not specifically limited in the embodiment of the present application.
  • the above-mentioned adaptive weight value may be applied only to the weighted average of the luminance components of the two reference image blocks, and the weighted average of the chrominance components still uses the uniform weighting method.
  • the bidirectional inter prediction apparatus may also determine that only a weight value of a pixel in a region of the first reference image block that is close to the first reference template is w0, and that is in a region of the second reference image block that is adjacent to the second reference template.
  • the weight value of the pixel is w1
  • the weight value of the pixel in the remaining regions of the first reference image block and the second reference image block is still 1/2.
  • the bidirectional inter prediction device performs weighting according to the weight of the pixel to obtain a corresponding pixel in the predicted image block.
  • the area close to the template is, for example, an area that satisfies max(x, y) ⁇ k ⁇ min(M, H), where max( ⁇ ) is a larger value operation and min( ⁇ ) is a smaller value operation.
  • the length of the current image block is the same as the length of the reference image block, and the width of the current image block is the same as the width of the reference image block.
  • the bidirectional inter prediction apparatus may further determine the predicted image block P C according to the following formula (14).
  • FIG. 6A and FIG. 6B also schematically show the flow of determining the predicted image block according to the method shown in FIG. 5 in the bidirectional inter prediction apparatus in the embodiment of the present application.
  • the bidirectional inter prediction apparatus determines a difference for characterizing the degree of difference between the two difference values based on a difference value between the current template and the first reference template and a difference value between the current template and the second reference template, and And determining, according to the preset range of the gap, a weight value corresponding to the first reference image block and a weight value corresponding to the second reference image block, that is, the bidirectional inter prediction device determines the weight value of the block level.
  • the bidirectional inter prediction method provided by the embodiment of the present application effectively improves the similarity between the current image block and the predicted image block.
  • the bidirectional inter prediction apparatus in the embodiment of the present application may determine the first reference image block and the second reference image according to the first reference frame index, the second reference frame index, the first motion vector, and the second motion vector.
  • the block may also determine the first reference image block and the second reference image block according to the search template.
  • the bidirectional inter prediction apparatus determines the first reference image block and the second reference image according to the search template.
  • the bidirectional inter prediction apparatus needs to first determine the search template.
  • the bidirectional inter prediction apparatus determines the search template based on the first image block in the first reference frame and the second image block in the second reference frame.
  • the bidirectional inter prediction apparatus may select, from the preset N pairs of weight values, a pair of weight values corresponding to the first image block and a weight value corresponding to the second image block, and compare the first image according to the weight value.
  • the block and the second image block are weighted to determine a search template; and the template of the first image block (ie, the first target template) and the template of the second image block (ie, the second target template) are respectively calculated with the current template.
  • a difference value and determining, according to the difference value, a weight value corresponding to the first image block and a weight value corresponding to the second image block, and weighting pixels of the first image block and pixels of the second image block according to the weight value, Determine the search template.
  • the bidirectional inter prediction apparatus first selects, from the preset N pairs of weight values, a pair of weight values corresponding to the first image block and a weight value corresponding to the second image block, and pairs the first image block according to the weight value.
  • the pixels and the pixels of the second image block are weighted to determine pixels of the search template, and then the first reference image block and the second reference image block are determined according to the search template for description.
  • S500 in the embodiment of the present application includes S500a to S500h, and S501 includes S501a.
  • the bidirectional inter prediction apparatus acquires a third reference frame index for indicating the first reference frame, a fourth reference frame index for indicating the second reference frame, a third motion vector, and a fourth motion vector.
  • the bidirectional inter prediction apparatus determines a first image block from the first reference frame according to the third reference frame index and the third motion vector, and according to the fourth reference frame index and the fourth motion vector, from the second reference frame. Determine the second image block.
  • the S500c, the bidirectional inter prediction device acquires the current template, the first target template, and the second target template.
  • the first target template includes adjacent and reconstructed pixel regions of the first image block
  • the second target template includes pixel regions adjacent to the second image block and reconstructed.
  • Tp0 is a template of a first image block, that is, Tp0 is a first target template
  • Tp1 is a template of a second image block, that is, Tp1 is a second target template
  • Tc is a current image block. Template, Tc is the current template.
  • the bidirectional inter prediction device weights the pixels of the first target template and the pixels of the second target template according to each pair of preset weight values according to N (N ⁇ 2) to the preset weight value, to obtain N composites. Space template.
  • the weight value corresponding to the first image block is wp0
  • the weight value corresponding to the second image block is wp1
  • the weights are ⁇ 1/6,5/6 ⁇ , ⁇ 2/6,4/6 ⁇ , ⁇ 3/6,3/6 ⁇ , ⁇ 4/6,2/6 ⁇ , ⁇ 5/6, 1/6 ⁇ .
  • the bidirectional inter prediction apparatus weights the pixels of the first target template and the pixels of the second target template according to each of the pair of preset weight values to obtain five composite space templates.
  • the bidirectional inter prediction apparatus may calculate a difference value between the current template and the first target template, and a difference value between the current template and the second target template by using the method shown in S502, so that N pairs are selected according to the two difference values. Preset weight value.
  • the weight value corresponding to the first image block is wp0
  • the weight value corresponding to the second image block is wp1
  • the preset weight values are ⁇ 1/6,5/6 ⁇ , ⁇ 2/6,4/6 ⁇ , ⁇ 3/6,3/6 ⁇ , ⁇ 4/6,2/6 ⁇ , ⁇ 5/6 , 1/6 ⁇ . If the difference value between the current template and the first target template is smaller than the difference between the current template and the second target template, the weight value corresponding to the first image block should be greater than or equal to the weight value corresponding to the second image block.
  • the bidirectional inter prediction only needs to use the following three pairs of preset weight values ⁇ 3/6, 3/6 ⁇ , ⁇ 4/6, 2/6 ⁇ , ⁇ 5/6, 1/6 ⁇ for the pixels and the second of the first target template.
  • the pixels of the target template are weighted to generate three different composite spatial templates.
  • the five pairs of preset weight values may also be ⁇ 2/8, 6/8 ⁇ , ⁇ 3/8, 5/8 ⁇ , ⁇ 4/8, 4/8 ⁇ , ⁇ 5/8, 3 /8 ⁇ , ⁇ 6/8, 2/8 ⁇ .
  • the S500e the bidirectional inter prediction device calculates a difference value between the current template and each of the synthesized spatial templates.
  • the bidirectional inter prediction apparatus may calculate a difference value between the current template and each of the synthesized spatial templates according to the SAD, or calculate a difference value between the current template and each of the synthesized spatial templates according to the SATD, and may also calculate according to the MR-SAD.
  • the value of the difference between the current template and each of the composite space templates is not specifically limited in this embodiment of the present application.
  • the bidirectional inter prediction device determines a composite space template with the smallest difference value, and determines a pair of preset weight values corresponding to the composite space template as a weight value corresponding to the first image block and a weight value corresponding to the second image block. .
  • the bidirectional inter prediction device weights the pixels of the first image block and the pixels of the second image block according to the weight value of the first image block and the weight value corresponding to the second image block, to obtain a search template.
  • the bidirectional inter prediction apparatus can determine the search template S C according to the following formula (15).
  • wp0 is a weight value corresponding to the first reference image block
  • wp1 is a weight value corresponding to the second reference image block
  • P 0 is a current image block obtained according to the third motion vector (dx2, dy2).
  • P 1 being a second image block of the corresponding position in the second reference frame obtained by the current image block according to the fourth motion vector (dx3, dy3).
  • the interpolation method herein may be a sub-pixel interpolation method in the H.265 standard, or may be a linear interpolation method, or may be another interpolation method, which is not specifically limited in the embodiment of the present application.
  • the bidirectional inter prediction apparatus determines the first reference image block according to the search template and the first reference frame, and determines the second reference image block according to the search template and the second reference frame.
  • the bidirectional inter prediction apparatus selects an image block having the smallest difference value from the search template in the window area of the preset size centering on the first image block in the first reference frame according to the search template obtained in S500g. And determining the image block as the first reference image block.
  • the bidirectional inter prediction device may calculate the difference value according to SAD, SATD or MR-SAD.
  • the bidirectional inter prediction device selects an image block having the smallest difference value from the search template in the window region of the preset size centered on the second image block in the second reference frame according to the search template obtained in S500g. And determining the image block as a second reference image block.
  • the bidirectional inter prediction apparatus acquires the first reference template and the second reference template.
  • FIG. 8 also schematically shows the flow of determining the predicted image block according to the method shown in FIG. 7 in the bidirectional inter prediction apparatus in the embodiment of the present application.
  • the bidirectional inter prediction apparatus determines the first reference image block and the second reference image block based on the search template, such that, based on the embodiment illustrated in FIG. 5, the present embodiment improves the current image block from the predicted image block.
  • the degree of similarity is the degree of similarity.
  • the difference between the template of the first image block (ie, the first target template) and the template of the second image block (ie, the second target template) and the current template is calculated for the bidirectional inter prediction device, and is determined according to the difference value.
  • a weight value corresponding to the first image block and a weight value corresponding to the second image block, and weighting pixels of the first image block and pixels of the second image block according to the weight value to determine a search template, and further determining according to the search template The first reference image block and the second reference image block are described.
  • the bidirectional inter prediction apparatus in the embodiment of the present application does not execute S500d to S500f, but performs S500x to S500y after executing S500c, and continues after S500y. Execute S500g.
  • the S500x, the bidirectional inter prediction device calculates a third difference value between the current template and the first target template, and a fourth difference value between the current template and the second target template.
  • the method for calculating the third difference value and the fourth difference value by the bidirectional inter prediction device may refer to the foregoing S502, and details are not described herein again.
  • the bidirectional inter prediction device determines, according to the third difference value and the fourth difference value, a weight value corresponding to the first image block and a weight value corresponding to the second image block.
  • the bidirectional inter prediction apparatus assigns a larger weight value to the first image block. Conversely, if the fourth difference value is much smaller than the third difference value, the bidirectional inter prediction apparatus assigns a larger weight value to the second image block.
  • the difference value and the weight value should be non-strict monotonically increasing relationships.
  • the third difference value be S2 and the fourth difference value be S3, where w2 is the weight value corresponding to the first image block, and w3 is the weight value corresponding to the second image block.
  • f(x, y) can be any of the following formulas:
  • the weight value corresponding to the first image block determined by the bidirectional inter prediction apparatus in S500y and the weight value corresponding to the second image block may be quantized to be 1/3, 1/16, ..., 1/256.
  • the embodiment of the present application does not specifically limit this.
  • the method for determining the weight value corresponding to the first image block and the weight value corresponding to the second image block by the bidirectional inter prediction device may refer to the description of S503 above, and details are not described herein again.
  • the bidirectional inter prediction apparatus in this embodiment also determines the first reference image block and the second reference image block based on the search template, and thus, based on the embodiment shown in FIG. 5, the present embodiment The degree of similarity between the current image block and the predicted image block is increased.
  • the embodiment of the present application provides a bidirectional inter prediction apparatus, which is used to perform the steps performed by the bidirectional inter prediction apparatus in the above bidirectional inter prediction method.
  • the bidirectional inter prediction apparatus provided by the embodiment of the present application may include a module corresponding to the corresponding step.
  • the embodiment of the present application may divide the function module into the bidirectional inter prediction device according to the foregoing method example.
  • each function module may be divided according to each function, or two or more functions may be integrated into one processing module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the division of modules in the embodiments of the present application is schematic, and is only a logical function division, and may be further divided in actual implementation.
  • FIG. 10 shows a possible structural diagram of the bidirectional inter prediction apparatus involved in the above embodiment in the case where the respective functional modules are divided by respective functions.
  • the bidirectional inter prediction apparatus 10 includes a determination unit 100, an acquisition unit 101, a calculation unit 102, and a weighting unit 103.
  • the determining unit 100 is for supporting the bidirectional inter prediction apparatus 10 to perform S500, S503, S500f, S500h, S500y, etc. in the above embodiments, and/or other processes for the techniques described herein.
  • the obtaining unit 101 is for supporting the bidirectional inter prediction apparatus 10 to execute S501, S500a, S500c, etc. in the above embodiments, and/or other processes for the techniques described herein.
  • the computing unit 102 is configured to support the bi-directional inter prediction apparatus 10 to perform S502, S500e, S500x, etc. in the above-described embodiments, and/or other processes for the techniques described herein.
  • the weighting unit 103 is for supporting the bidirectional inter prediction apparatus 10 to perform S504, S500d, S500g, etc. in the above embodiments, and/or other processes for the techniques described herein.
  • the bidirectional inter prediction apparatus 10 provided by the embodiment of the present application includes, but is not limited to, the foregoing module.
  • the bidirectional inter prediction apparatus may further include a storage unit 104.
  • the storage unit 104 can be used to store program codes and data of the bi-directional inter prediction device 10.
  • the bidirectional inter prediction apparatus 11 includes a processing module 110 and a communication module 111.
  • the processing module 110 is configured to control and manage the actions of the bi-directional inter prediction device 11, for example, perform the steps performed by the determining unit 100, the computing unit 102, and the weighting unit 103, and/or other techniques for performing the techniques described herein. process.
  • the communication module 111 is for supporting interaction between the bidirectional inter prediction apparatus 11 and other devices, for example, performing the steps performed by the acquisition unit 101 described above.
  • the bidirectional inter prediction apparatus 11 may further include a storage module 112 for storing program codes and data of the bidirectional inter prediction apparatus 11, for example, storing contents held by the storage unit 104.
  • the processing module 110 may be a processor or a controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), and an application-specific integrated circuit (Application-Specific). Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure.
  • the processor may also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like.
  • the communication module 111 can be a transceiver, an RF circuit or a communication interface, or the like.
  • the storage module 112 can be a memory.
  • the bidirectional inter prediction apparatus 10 and the bidirectional inter prediction apparatus 11 can perform the bidirectional inter prediction method shown in any of the above-described FIGS. 5, 7, and 9, the bidirectional inter prediction apparatus 10 and the bidirectional interframe
  • the prediction device 11 may specifically be a video encoding device, a video decoding device, or other device having a video codec function.
  • the bidirectional inter prediction apparatus 10 and the bidirectional inter prediction apparatus 11 can be used for both image prediction in the encoding process and image prediction in the decoding process.
  • the application also provides a terminal, the terminal comprising: one or more processors, a memory, and a communication interface.
  • the memory, communication interface is coupled to one or more processors; the memory is for storing computer program code, and the computer program code includes instructions for performing bidirectional inter prediction of embodiments of the present application when one or more processors execute the instructions method.
  • the terminals here can be video display devices, smart phones, laptops and other devices that can process video or play video.
  • the present application also provides a video encoder, including a non-volatile storage medium, and a central processing unit, the non-volatile storage medium storing an executable program, the central processing unit and the non-volatile storage The medium is connected, and the executable program is executed to implement the bidirectional inter prediction method of the embodiment of the present application.
  • the present application also provides a video decoder including a nonvolatile storage medium, and a central processing unit, the nonvolatile storage medium storing an executable program, the central processing unit and the nonvolatile storage The medium is connected, and the executable program is executed to implement the bidirectional inter prediction method of the embodiment of the present application.
  • the present application further provides a decoder, which includes a bidirectional inter prediction apparatus (a bidirectional inter prediction apparatus 10 or a bidirectional inter prediction apparatus 11) and a reconstruction module in the embodiment of the present application, wherein the reconstruction module A reconstructed pixel value of the current image block is determined for pixels of the predicted image block obtained from the bidirectional inter prediction device.
  • a bidirectional inter prediction apparatus a bidirectional inter prediction apparatus 10 or a bidirectional inter prediction apparatus 11
  • the reconstruction module A reconstructed pixel value of the current image block is determined for pixels of the predicted image block obtained from the bidirectional inter prediction device.
  • the present application further provides an encoder, which includes a bidirectional inter prediction apparatus (a bidirectional inter prediction apparatus 10 or a bidirectional inter prediction apparatus 11) and a reconstruction module in the embodiment of the present application, wherein the reconstruction module And determining a reconstructed pixel value of the current image block according to a pixel value of the predicted image block obtained by the bidirectional inter prediction device.
  • a bidirectional inter prediction apparatus a bidirectional inter prediction apparatus 10 or a bidirectional inter prediction apparatus 11
  • the reconstruction module And determining a reconstructed pixel value of the current image block according to a pixel value of the predicted image block obtained by the bidirectional inter prediction device.
  • Another embodiment of the present application also provides a computer readable storage medium including one or more program codes, the one or more programs including instructions, when a processor in a terminal is executing the program code
  • the terminal performs a bidirectional inter prediction method as shown in any of FIGS. 5, 7, and 9.
  • a computer program product comprising computer executable instructions stored in a computer readable storage medium; at least one processor of the terminal Reading the storage medium to read the computer to execute the instruction, and the at least one processor executing the computer to execute the instruction to cause the terminal to perform bidirectional inter prediction in the bidirectional inter prediction method shown in any of FIG. 5, FIG. 7 and FIG. The steps of the device.
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • a software program it may occur in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center Transfer to another website site, computer, server, or data center by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL), or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media.
  • the usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (such as a solid state disk (SSD)).
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules or units is only a logical function division.
  • there may be another division manner for example, multiple units or components may be used.
  • the combination may be integrated into another device, or some features may be ignored or not performed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed to multiple different places. . Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a readable storage medium.
  • the technical solution of the embodiments of the present application may be embodied in the form of a software product in the form of a software product in essence or in the form of a contribution to the prior art, and the software product is stored in a storage medium.
  • a number of instructions are included to cause a device (which may be a microcontroller, chip, etc.) or a processor to perform all or part of the steps of the methods described in various embodiments of the present application.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例公开了一种双向帧间预测方法及装置，涉及视频图像编解码技术领域，自适应确定出双向加权预测的权重值，提高了双向加权预测的预测性能。该方法包括：根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块；获取当前模板、第一参考模板和第二参考模板；计算当前模板与第一参考模板之间的第一差异值，以及当前模板与第二参考模板之间的第二差异值；确定用于表征第一差异值与第二差异值的差异程度的差距，并根据该差距归属的预设范围，确定第一参考图像块的权重值和第二参考图像块的权重值；根据第一参考图像块的权重值和第二参考图像块的权重值，对第一参考图像块和第二参考图像块进行加权，得到预测图像块。

Description

一种双向帧间预测方法及装置
本申请要求于2018年02月06日提交中国专利局、申请号为201810118179.9、发明名称为“一种双向帧间预测方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及视频图像编解码技术领域，尤其涉及一种双向帧间预测方法及装置。
背景技术
在视频编码中，运动补偿的帧间预测技术对编码效率的提高有非常重要的作用。
对于当前图像块而言，可以仅根据一个参考图像块生成当前图像块的预测图像块(即单向预测)，也可以根据两个参考图像块生成当前图像块的预测图像块(即双向帧间预测)，上述两个参考图像块可来自于同一个参考帧或者不同的参考帧。一般的，在双向帧间预测的过程中，将两个参考图像块根据各自的权重值进行加权，以生成当前图像块的预测图像块。
在现有的视频编码标准中，双向帧间预测是根据条带级的权重值对两个参考图像块进行加权，以生成预测图像块，或者使用默认权重值(1/2，1/2)对两个参考图像块进行均匀加权，以生成预测图像块。但是，在两个参考图像块与当前图像块的差异值有明显差别的场景中，使用现有方法生成的预测图像块的准确率较低，且预测性能较低。
发明内容
本申请实施例提供一种双向帧间预测方法及装置，能够解决目前获取到的预测图像块的准确率较低、预测性能低的问题。
为达到上述目的，本申请实施例采用如下技术方案：
第一方面，提供一种双向帧间预测方法，该双向帧间预测方法为：
在根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块后，获取当前模板(当前模板包括与当前图像块相邻、且已重建的像素区域)、第一参考模板(第一参考模板包括与第一参考图像块相邻、且已重建的像素区域)和第二参考模板(第二参考模板包括与第二参考图像块相邻、且已重建的像素区域)，这里，当前模板与当前图像块的关系和参考模板与参考图像块的关系相同；然后，计算当前模板与第一参考模板之间的第一差异值，以及当前模板与第二参考模板之间的第二差异值，并确定用于表征第一差异之与第二差异值的差异程度的差距，进而根据该差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值，这样，即可根据第一参考图像块对应的权重值和第二参考图像块对应的权重值，对第一参考图像块的像素和第二参考图像块的像素进行加权，得到预测图像块的像素。所述差距归属不同预设范围时，第一参考图像块对应的权重值不同，第二参考图像块对应的权重值不同。
本申请实施例提供的双向帧间预测方法确定出了第一参考图像块对应的权重值和第二参考图像块对应的权重值，即确定出了块级的权重值，相比于现有技术，本申请实施例提供的双向帧间预测方法有效的提高了当前图像块与预测图像块之间的相似程度，预测性能较高。
可选的，在本申请的一种可能的实现方式中，上述“确定差距”的方法为：根据下述公式确定差距：
D＝(S1 n+k)/(S0 n+k)
其中，D表示差距，S1表示第二差异值，S0表示第一差异值，n为第一预设常数，k为第二预设常数。
可选的，在本申请的另一种可能的实现方式中，上述“确定差距”的方法为：根据下述公式确定差距：
D＝S1 m-S0 m
其中，D表示差距，S1表示第二差异值，S0表示第一差异值，m为第三预设常数。
可选的，在本申请的另一种可能的实现方式中，上述“根据差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值”的方法为：从预设对应关系中查找与预设范围对应的权重值偏移量；预设对应关系包括多个预设差距范围以及每个预设差距范围对应的权重值偏移量，预设范围为多个预设差距范围中的一个；根据第一预设公式和权重值偏移量，确定第一参考图像块对应的权重值，并根据第二预设公式以及第一参考图像块对应的权重值，确定第二参考图像块对应的权重值；其中，第一预设公式为：w0＝1/2+Δw；第二预设公式为：w1＝1-w0。w0为第一参考图像块对应的权重值，Δw为权重值偏移量，w1为第二参考图像块对应的权重值。
可选的，在本申请的另一种可能的实现方式中，上述“根据差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值”的方法为：根据第三预设公式，确定与预设范围对应的权重值偏移量；根据第一预设公式和权重值偏移量，确定第一参考图像块对应的权重值，并根据第二预设公式和第一参考图像块对应的权重值，确定第二参考图像块对应的权重值；其中，第一预设公式为：w0＝1/2+Δw；第二预设公式为：w1＝1-w0；第三预设公式为：
Figure PCTCN2018119430-appb-000001
w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，Δw表示权重值偏移量，floor(·)为向下取整函数，P表示权重值的精度，o是第四预设常数，q为第五预设常数，i为第六预设常数，D min表示差距D的最小值。
可选的，在本申请的另一种可能的实现方式中，上述当前图像块的双向运动信息包括用于指示第一参考帧的第一参考帧索引、用于指示第二参考帧的第二参考帧索引、第一运动矢量以及第二运动矢量。
在这种情况下，上述“根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块”的方法为：获取第一参考帧索引、第二参考帧索引、第一运动矢量 和第二运动矢量；根据第一参考帧索引，从第一参考帧列表中获取第一参考帧，根据第二参考帧索引，从第二参考帧列表中获取第二参考帧；根据第一运动矢量和第一参考帧，确定第一参考图像块，并根据第二运动矢量和第二参考帧，确定第二参考图像块。
可选的，在本申请的另一种可能的实现方式中，上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量。
在这种情况下，上述“根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块”的方法为：获取第三参考帧索引、第四参考帧索引、第三运动矢量和第四运动矢量；根据第三参考帧索引和第三运动矢量，从第一参考帧中确定第一图像块，以及根据第四参考帧索引和第四运动矢量，从第二参考帧中确定第二图像块；该获取第一目标模板和第二目标模板，这里，第一目标模板包括第一图像块相邻、且已重建的像素区域，第二目标模板包括第二图像块相邻、且已重建的像素区域；根据第一目标模板和第二目标模板分别与当前模板之间的差异值，确定第一图像块对应的权重值和第二图像块对应的权重值；根据第一图像块的权要值和第二图像块对应的权重值，对第一图像块的像素和第二图像块的像素进行加权，得到搜索模板，并根据搜索模板，从第一参考帧中确定第一参考图像块，并从第二参考帧中确定第二参考图像块。
可以看出，本申请可以直接根据获取到的双向运动信息确定出第一参考图像块和第二参考图像块，也可以利用获取到的双向运动信息确定第一图像块和第二图像块，并根据第一图像块和第二图像块确定出搜索模板，进而根据该搜索模板从第一参考帧中确定出第一参考图像块，从第二参考帧中确定出第二参考图像块。利用搜索模板确定出第一参考图像块和第二参考图像块，进一步地提高了当前图像块与预测图像块之间的相似程度。
可选的，在本申请的另一种可能的实现方式中，上述“根据第一目标模板和第二目标模板分别与当前模板之间的差异值，确定第一图像块对应的权重值和第二图像块对应的权重值”的方法为：计算当前模板与第一目标模板之间的第三差异值，以及当前模板与第二目标模板之间的第四差异值，并根据第三差异值与第四差异值，确定第一图像块对应的权重值和第二图像块对应的权重值。
可选的，在本申请的另一种可能的实现方式中，上述“根据第一目标模板和第二目标模板分别与当前模板之间的差异值，确定第一图像块对应的权重值和第二图像块对应的权重值”的方法为：根据N(N≥2)对预设权重值，对第一目标模板的像素和第二目标模板的像素分别按照每一对预设权重值进行加权，得到N个合成空间模板，然后，计算当前模板与每一合成空间模板之间的差异值，这样，能够确定出差异值最小的合成空间模板，根据与该合成空间模板对应的一对预设权重值确定第一图像块对应的权重值和第二图像块对应的权重值。
本申请可以采用不同的方法确定第一图像块对应的权重值和第二图像块对应的权重值，本申请实施例对此不作具体限定。
第二方面，提供一种双向帧间预测装置，该双向帧间预测装置包括确定单元、获 取单元、计算单元以及加权单元。
具体的，上述确定单元，用于根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块。上述获取单元，用于获取当前模板、第一参考模板和第二参考模板，当前模板包括与上述确定单元确定的当前图像块相邻、且已重建的像素区域，第一参考模板包括与上述确定单元确定的第一参考图像块相邻、且已重建的像素区域，第二参考模板包括与上述确定单元确定的第二参考图像块相邻、且已重建的像素区域；当前模板与当前图像块的位置关系、第一参考模板与第一参考图像块的位置关系、以及第二参考模板与第二参考图像块的位置关系均相同。上述计算单元，用于计算上述获取单元获取到的当前模板与第一参考模板之间的第一差异值，以及当前模板与第二参考模板之间的第二差异值。上述确定单元，还用于确定差距，该差距用于表征上述计算单元计算出的第一差异值与第二差异值的差异程度，以及根据差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值，所述差距归属不同预设范围时，第一参考图像块对应的权重值不同，第二参考图像块对应的权重值不同。上述加权单元，用于根据上述确定单元确定出的第一参考图像块对应的权重值和第二参考图像块对应的权重值，对第一参考图像块的像素和第二参考图像块的像素进行加权，得到预测图像块的像素。
可选的，在本申请的一种可能的实现方式中，上述确定单元具体用于：根据公式D＝(S1 n+k)/(S0 n+k)确定差距，其中，D表示差距，S1表示第二差异值，S0表示第一差异值，n为第一预设常数，k为第二预设常数。
可选的，在本申请的另一种可能的实现方式中，上述确定单元具体用于：根据公式D＝S1 m-S0 m确定差距，其中，D表示差距，S1表示第二差异值，S0表示第一差异值，m为第三预设常数。
可选的，在本申请的另一种可能的实现方式中，上述确定单元具体用于：从预设对应关系中查找与预设范围对应的权重值偏移量；预设对应关系包括多个预设差距范围以及每个预设差距范围对应的权重值偏移量，预设范围为多个预设差距范围中的一个；根据第一预设公式和权重值偏移量，确定第一参考图像块对应的权重值，并根据第二预设公式以及第一参考图像块对应的权重值，确定第二参考图像块对应的权重值；其中，第一预设公式为：w0＝1/2+Δw；第二预设公式为：w1＝1-w0；w0为第一参考图像块对应的权重值，Δw为权重值偏移量，w1为第二参考图像块对应的权重值。
可选的，在本申请的另一种可能的实现方式中，上述确定单元具体用于：根据第三预设公式，确定与预设范围对应的权重值偏移量；根据第一预设公式和权重值偏移量，确定第一参考图像块对应的权重值，并根据第二预设公式和第一参考图像块对应的权重值，确定第二参考图像块对应的权重值；其中，第一预设公式为：w0＝1/2+Δw；第二预设公式为：w1＝1-w0；第三预设公式为：
Figure PCTCN2018119430-appb-000002
w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，Δw表示权重值偏移量，floor(·)为向下取整函数，P表示权重值的精度，o是第四预设常数，q为第五预设常数，i为第六预设常数，D min表示差距D的最小值。
可选的，在本申请的另一种可能的实现方式中，上述当前图像块的双向运动信息包括用于指示第一参考帧的第一参考帧索引、用于指示第二参考帧的第二参考帧索引、第一运动矢量以及第二运动矢量。
相应的，上述获取单元，还用于获取第一参考帧索引、第二参考帧索引、第一运动矢量和第二运动矢量，以及根据第一参考帧索引，从第一参考帧列表中获取第一参考帧，并根据第二参考帧索引，从第二参考帧列表中获取第二参考帧。上述确定单元，具体用于根据上述获取单元获取到的第一运动矢量和第一参考帧，确定第一参考图像块，并根据第二运动矢量和第二参考帧，确定第二参考图像块。
可选的，在本申请的另一种可能的实现方式中，上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量。
相应的，上述获取单元，还用于获取第三参考帧索引、第四参考帧索引、第三运动矢量和第四运动矢量，以及根据第三参考帧索引和第三运动矢量，从第一参考帧中确定第一图像块，以及根据第四参考帧索引和第四运动矢量，从第二参考帧中确定第二图像块，以及获取第一目标模板和第二目标模板，第一目标模板包括第一图像块相邻、且已重建的像素区域，第二目标模板包括第二图像块相邻、且已重建的像素区域。上述确定单元，具体用于根据上述获取单元获取到的第一目标模板和第二目标模板分别与当前模板之间的差异值，确定第一图像块对应的权重值和第二图像块对应的权重值。上述加权单元，还用于根据上述确定单元确定出的第一图像块的权要值和第二图像块对应的权重值，对第一图像块的像素和第二图像块的像素进行加权，得到搜索模板。上述确定单元，具体用于根据上述加权单元得到的搜索模板，从第一参考帧中确定第一参考图像块，并从第二参考帧中确定第二参考图像块。
可选的，在本申请的另一种可能的实现方式中，在上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量的情况下，上述计算单元，还用于计算当前模板与第一目标模板之间的第三差异值，以及当前模板与第二目标模板之间的第四差异值。上述确定单元，具体用于根据计算单元计算出的第三差异值与第四差异值，确定第一图像块对应的权重值和第二图像块对应的权重值。
可选的，在本申请的另一种可能的实现方式中，在上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量的情况下，上述加权单元，还用于根据N对预设权重值，对第一目标模板的像素和第二目标模板的像素分别按照每一对预设权重值进行加权，得到N个合成空间模板，N≥2。上述计算单元，还用于计算当前模板与上述加权单元得到的每一合成空间模板之间的差异值。上述确定单元，具体用于确定上述计算单元计算出的差异值最小的合成空间模板，根据与该合成空间模板对应的一对预设权重值确定第一图像块对应的权重值和第二图像块对应的权重值。
第三方面，提供一种终端，该终端包括：一个或多个处理器、存储器、通信接口。该存储器、通信接口与一个或多个处理器耦合；存储器用于存储计算机程序代码，计算机程序代码包括指令，当一个或多个处理器执行指令时，终端执行如上述第一方面 及其任意一种可能的实现方式所述的双向帧间预测方法。
具体的，上述处理器用于：根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块；通过上述通信接口获取当前模板、第一参考模板和第二参考模板，当前模板包括与当前图像块相邻、且已重建的像素区域，第一参考模板包括与第一参考图像块相邻、且已重建的像素区域，第二参考模板包括与第二参考图像块相邻、且已重建的像素区域，当前模板与当前图像块的位置关系、第一参考模板与第一参考图像块的位置关系、以及第二参考模板与第二参考图像块的位置关系均相同；计算当前模板与第一参考模板之间的第一差异值，以及当前模板与第二参考模板之间的第二差异值；确定差距，该差距用于表征上述计算单元计算出的第一差异值与第二差异值的差异程度，以及根据差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值，所述差距归属不同预设范围时，第一参考图像块对应的权重值不同，第二参考图像块对应的权重值不同；根据第一参考图像块对应的权重值和第二参考图像块对应的权重值，对第一参考图像块的像素和第二参考图像块的像素进行加权，得到预测图像块的像素。
可选的，在本申请的一种可能的实现方式中，上述处理器具体用于：根据公式D＝(S1 n+k)/(S0 n+k)确定差距，其中，D表示差距，S1表示第二差异值，S0表示第一差异值，n为第一预设常数，k为第二预设常数。
可选的，在本申请的另一种可能的实现方式中，上述处理器具体用于：根据公式D＝S1 m-S0 m确定差距，其中，D表示差距，S1表示第二差异值，S0表示第一差异值，m为第三预设常数。
可选的，在本申请的另一种可能的实现方式中，上述确定单元具体用于：从预设对应关系中查找与预设范围对应的权重值偏移量；预设对应关系包括多个预设差距范围以及每个预设差距范围对应的权重值偏移量，预设范围为多个预设差距范围中的一个；根据第一预设公式和权重值偏移量，确定第一参考图像块对应的权重值，并根据第二预设公式以及第一参考图像块对应的权重值，确定第二参考图像块对应的权重值；其中，第一预设公式为：w0＝1/2+Δw；第二预设公式为：w1＝1-w0；w0为第一参考图像块对应的权重值，Δw为权重值偏移量，w1为第二参考图像块对应的权重值。
可选的，在本申请的另一种可能的实现方式中，上述处理器具体用于：根据第三预设公式，确定与预设范围对应的权重值偏移量；根据第一预设公式和权重值偏移量，确定第一参考图像块对应的权重值，并根据第二预设公式和第一参考图像块对应的权重值，确定第二参考图像块对应的权重值；其中，第一预设公式为：w0＝1/2+Δw；第二预设公式为：w1＝1-w0；第三预设公式为：
Figure PCTCN2018119430-appb-000003
w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，Δw表示权重值偏移量，floor(·)为向下取整函数，P表示权重值的精度，o是第四预设常数，q为第五预设常数，i为第六预设常数，D min表示差距D的最小值。
可选的，在本申请的另一种可能的实现方式中，上述当前图像块的双向运动信息包括用于指示第一参考帧的第一参考帧索引、用于指示第二参考帧的第二参考帧索引、 第一运动矢量以及第二运动矢量。
相应的，上述处理器还用于：获取第一参考帧索引、第二参考帧索引、第一运动矢量和第二运动矢量，以及根据第一参考帧索引，从第一参考帧列表中获取第一参考帧，并根据第二参考帧索引，从第二参考帧列表中获取第二参考帧；根据获取到的第一运动矢量和第一参考帧，确定第一参考图像块，并根据第二运动矢量和第二参考帧，确定第二参考图像块。
可选的，在本申请的另一种可能的实现方式中，上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量。
相应的，上述处理器还用于：获取第三参考帧索引、第四参考帧索引、第三运动矢量和第四运动矢量，以及根据第三参考帧索引和第三运动矢量，从第一参考帧中确定第一图像块，以及根据第四参考帧索引和第四运动矢量，从第二参考帧中确定第二图像块，以及获取第一目标模板和第二目标模板，第一目标模板包括第一图像块相邻、且已重建的像素区域，第二目标模板包括第二图像块相邻、且已重建的像素区域；根据获取到的第一目标模板和第二目标模板分别与当前模板之间的差异值，确定第一图像块对应的权重值和第二图像块对应的权重值；根据第一图像块的权要值和第二图像块对应的权重值，对第一图像块的像素和第二图像块的像素进行加权，得到搜索模板；根据搜索模板，从第一参考帧中确定第一参考图像块，并从第二参考帧中确定第二参考图像块。
可选的，在本申请的另一种可能的实现方式中，在上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量的情况下，上述处理器还用于：计算当前模板与第一目标模板之间的第三差异值，以及当前模板与第二目标模板之间的第四差异值；根据计算出的第三差异值与第四差异值，确定第一图像块对应的权重值和第二图像块对应的权重值。
可选的，在本申请的另一种可能的实现方式中，在上述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量的情况下，上述处理器还用于：根据N对预设权重值，对第一目标模板的像素和第二目标模板的像素分别按照每一对预设权重值进行加权，得到N个合成空间模板，N≥2；计算当前模板与每一合成空间模板之间的差异值；确定计算出的差异值最小的合成空间模板，根据与该合成空间模板对应的一对预设权重值确定第一图像块对应的权重值和第二图像块对应的权重值。
第四方面，提供一种视频编码器，包括非易失性存储介质以及中央处理器，所述非易失性存储介质存储有可执行程序，所述中央处理器与所述非易失性存储介质连接，并执行所述可执行程序以实现所述第一方面及其任意一种可能的实现方式所述的双向帧间预测方法。
第五方面，提供一种视频解码器，包括非易失性存储介质以及中央处理器，所述非易失性存储介质存储有可执行程序，所述中央处理器与所述非易失性存储介质连接，并执行所述可执行程序以实现所述第一方面及其任意一种可能的实现方式所述的双向 帧间预测方法。
第六方面，提供一种编码器，所述编码器包括上述第二方面中的双向帧间预测装置以及重建模块，其中，所述重建模块用于根据所述双向帧间预测装置得到的预测图像块的像素确定当前图像块的重建像素值。
第七方面，提供一种解码器，所述解码器包括上述第二方面中的双向帧间预测装置以及重建模块，其中，所述重建模块用于根据所述双向帧间预测装置得到的预测图像块的像素确定当前图像块的重建像素值。
第八方面，提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当所述指令在上述第三方面所述的终端上运行时，使得所述终端执行如上述第一方面及其任意一种可能的实现方式所述的双向帧间预测方法。
第九方面，提供一种包含指令的计算机程序产品，当该计算机程序产品在上述第三方面所述的终端上运行时，使得所述终端执行如上述第一方面及其任意一种可能的实现方式所述的双向帧间预测方法。
可选的，在上述任一方面及其任一可能的实现方式中，上述差异值均可以根据误差绝对值和(Sum of Absolute Differences，SAD)、误差变换系数绝对值之和(Sum of Absolute Transformed Difference，SATD)或去均值的误差绝对值之和(Mean-Removal Sum of Absolute Differences，MR-SAD)计算得到。
在本申请中，上述双向帧间预测装置的名字对设备或功能模块本身不构成限定，在实际实现中，这些设备或功能模块可以以其他名称出现。只要各个设备或功能模块的功能和本申请类似，属于本申请权利要求及其等同技术的范围之内。
本申请中第二方面到第九方面，及其各种实现方式的具体描述，可以参考第一方面及其各种实现方式中的详细描述；并且，第二方面到第九方面，及其各种实现方式的有益效果，可以参考第一方面及其各种实现方式中的有益效果分析，此处不再赘述。
本申请的这些方面或其他方面在以下的描述中会更加简明易懂。
附图说明
图1为本申请实施例中CTU的结构示意图；
图2为本申请实施例中图像块的模板的结构示意图；
图3为本申请实施例提供的视频传输系统的结构示意图；
图4A为本申请实施例提供的视频编解码装置的结构示意图；
图4B为本申请实施例提供的视频编解码系统的结构示意图；
图5为本申请实施例提供的双向帧间预测方法流程示意图一；
图6A为本申请实施例提供的当前模板、第一参考模板和第二参考模板的结构示意图一；
图6B为本申请实施例提供的当前模板、第一参考模板和第二参考模板的结构示意图二；
图7为本申请实施例提供的双向帧间预测方法流程示意图二；
图8为本申请实施例提供的当前图像块、搜索模板以及预测图像块的结构示意图；
图9为本申请实施例提供的双向帧间预测方法流程示意图三；
图10为本申请实施例提供的双向帧间预测装置的结构示意图；
图11为本申请实施例提供的双向帧间预测装置的结构示意图。
具体实施方式
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”和“第四”等是用于区别不同对象，而不是用于限定特定顺序。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
为了方便理解本申请实施例，首先在此介绍本申请实施例涉及到的相关要素。
编码树单元(Coding Tree Unit，CTU)：一幅图像由多个CTU构成，一个CTU通常对应于一个方形图像区域，如图1所示，图像10由多个CTU构成(包括CTU A、CTU B、CTU C等)。与某一CTU对应的编码信息包含与该CTU对应的方形图像区域中的像素的亮度值和/或色度值。此外，与某一CTU对应的编码信息还可以包含语法元素，这些语法元素指示如何将该CTU划分成至少一个编码单元(Coding Unit，CU)，以及解码每个CU以得到重建图像的方法。
一个CTU对应的图像区域可以包括64×64、128×128或256×256个像素。在一个示例中，64×64个像素的CTU包含由64列、每列64个像素的矩形像素点阵，每个像素包含亮度分量和/或色度分量。
可以理解的是，CTU也可以对应矩形图像区域或者其它形状的图像区域，一个CTU对应的图像区域也可以是水平方向的像素点的数量与竖直方向的像素点数量不同的图像区域，例如包括64×128个像素。
CU：通常对应于一个A×B的矩形区域，A为矩形的宽，B为矩形的高，其中，本申请实施例中涉及到的宽是指图1示出的二维直角坐标系XoY中沿X轴方向(水平方向)的长度，高是指图1示出的二维直角坐标系XoY中沿Y轴方向(竖直方向)的长度。这里，A和B的取值可以相同，也可以不同。A和B的取值通常为2的整数次幂，例如：256、128、64、32、16、8或4等。
一个CU可通过解码处理得到一个A×B的矩形区域的重建图像块，解码处理通常包括预测(prediction)、反量化(dequantization)、反变换(inversetransform)等处理，产生预测图像和残差图像，预测图像和残差图像叠加后得到重建图像块。多个重建图像块可以得到最终的重建图像。
条带(Slice)：由一系列的按照光栅扫描顺序排列的CTU组成。一幅图像可以分解为一个或若干个条带，每个条带各包括一组CTU。
帧间预测：指以编码图像块/解码图像块为单位，利用当前帧与其参考帧之间的相关性完成的预测，当前帧可以存在一个或多个参考帧。具体的，根据当前图像块的参考帧中的像素，生成当前图像块的预测图像块。
一般的，对于当前图像块而言，可以仅根据一个参考图像块生成当前图像块的预测图像块，也可以根据两个参考图像块生成当前图像块的预测图像块。上述根据一个参考图像块生成当前图像块的预测图像块称为单向预测，上述根据两个参考图像块生成当前图像块的预测图像块称为双向帧间预测。双向帧间预测中的两个参考图像块可 来自于同一个参考帧或者不同的参考帧。
可选的，双向帧间预测可以是指利用当前视频帧与在其之前编码且在其之前播放的视频帧之间的相关性，和当前视频帧与在其之前编码且在其之后播放的视频帧之间的相关性进行的帧间预测。
可以看出，上述双向帧间预测涉及两个方向的帧间预测，一般称为：前向帧间预测和后向帧间预测。前向帧间预测是指利用当前视频帧与在其之前编码且在其之前播放的视频帧之间的相关性进行的帧间预测。后向帧间预测是指利用当前视频帧与在其之前编码且在其之后播放的视频帧之间的相关性进行的帧间预测。
前向帧间预测对应前向参考帧列表L0，后向帧间预测对应后向参考帧列表L1，两个参考帧列表中所包含的参考帧的数量可以相同，也可以不同。
图像编码(image encoding)：将图像序列压缩成码流的处理过程。
图像解码(image decoding)：将码流按照特定的语法规则和处理方法恢复成重建图像的处理过程。
在大多数的编码框架中，视频序列包括一系列图像(picture)，图像被划分为至少一个条带，每个条带又被划分为图像块(block)。视频编码/解码以图像块为单位，可从图像的左上角位置开始从左到右、从上到下、一行一行进行编码/解码处理。这里，图像块可以为视频编解码标准H.264中的宏块(macro block，MB)，也可以为高效视频编码(High Efficiency Video Coding，HEVC)标准中的CU，本申请实施例对此不作具体限定。
本申请中，正在进行编码/解码处理的图像块称为当前图像块(current block)，当前图像块所在的图像称为当前帧。
当前帧可以为单向预测帧(P帧)，也可以为双向预测帧(B帧)。在当前帧为P帧的情况下，当前帧具有一个参考帧列表。在当前帧为B帧的情况下，当前帧具有两个参考帧列表，这两个列表通常分别称为L0和L1。每个参考帧列表均包含至少一个用作当前帧的参考帧的重建帧。参考帧用于为当前帧的帧间预测提供参考像素。
在当前帧中，与当前图像块相邻的(例如位于当前块的左侧、上侧或右侧)的图像块可能已经完成了编码/解码处理，得到了重建图像，它们称为重建图像块；重建图像块的编码模式、重建像素等信息是可以获得的(available)。
在当前帧进行编码/解码之前已经完成编码/解码处理的帧称为重建帧。
图像块的模板：包括该图像块相邻、且已重建的像素区域。其中，图像块和该图像块的模板之间可以没有重叠区。示例性的，图像块的模板包括与该图像块相邻、且位于该图像块某一方向(如上侧、左侧和右侧中的至少一个方向)的至少一行重建像素。
可选的，模板的形状可以是L型，也可以是其他形状，本申请对此不作具体限定。例如，如图2所示，图像块20周围相邻的L型像素区域20＇为图像块20的模板，位于图像块21左侧且与图像块21相邻的像素区域21＇为图像块21的模板，位于图像块22左侧、上测以及右侧、且与图像块22相邻的像素区域22＇为图像块22的模板。
运动矢量(Motion Vector，MV)：表示当前图像块与参考图像块之间的相对位移。
由于视频中连续的若干帧图像之间存在较强时间相关性，所以在进行视频编码时， 常利用各个帧之间的时间相关性来减少视频帧间的冗余，达到压缩数据的目的。目前，主要采用运动补偿的帧间预测技术对视频图像进行编码。
具体的，编码端在对当前帧中的当前图像块进行编码时，首先从视频图像已编码的帧中任意选取一个以上参考帧，并从所述参考帧中获取当前图像块对应的预测图像块，然后计算预测图像块与当前图像块之间的残差值，对该残差值进行量化编码；解码端在对当前帧中的当前图像块进行解码时，首先获取当前图像块对应的预测图像块，然后从接收到的码流中获取预测图像块与当前图像块的残差值，根据该残差值和预测块解码重构当前图像块。
视频中当前帧与其他帧之间的时间相关性不仅表现在当前帧与在其之前编码的帧之间存在时间相关性，也表现在当前帧与在其之后编码的帧之间存在时间相关性。基于此，在进行视频编码时，可以考虑双向帧间预测，以得到较佳的编码效果。
在双向帧间预测的过程中，将两个参考图像块根据各自的权重值进行加权，以生成当前图像块的预测图像块。在现有的视频编码标准中，双向帧间预测是根据条带级的权重值对两个参考图像块进行加权，以生成预测图像块，或者使用默认权重值(1/2，1/2)对两个参考图像块进行均匀加权，以生成预测图像块。但是，在两个参考图像块与当前图像块的差异值有明显差别的场景中，使用现有方法生成的预测图像块与当前图像块的相似程度较低。
针对上述问题，本申请实施例提供一种双向帧间预测方法，在确定出第一参考图像块和第二参考图像块后，获取第一参考模板(第一参考模板包括与第一参考图像块相邻、且已重建的像素区域)、第二参考模板(第二参考模板包括与第二参考图像块相邻、且已重建的像素区域)以及当前模板(当前模板包括与当前图像块相邻、且已重建的像素区域)，然后，计算当前模板与第一参考模板之间的第一差异值，以及当前模板与第二参考模板之间的第二差异值，并确定用于表征第一差异之与第二差异值的差异程度的差距，进而根据该差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值，这样，即可根据第一参考图像块对应的权重值和第二参考图像块对应的权重值，对第一参考图像块的像素和第二参考图像块的像素进行加权，得到预测图像块的像素。本申请实施例中当前模板与当前图像块的位置关系、第一参考模板与第一参考图像块的位置关系、以及第二参考模板与第二参考图像块的位置关系均相同。所述差距归属不同预设范围时，第一参考图像块对应的权重值不同，第二参考图像块对应的权重值不同。
可以看出，本申请提供的双向帧间预测方法确定出了第一参考图像块对应的权重值和第二参考图像块对应的权重值，即确定出了块级的权重值，相比于现有技术，本申请实施例提供的双向帧间预测方法有效的提高了当前图像块与预测图像块之间的相似程度。
本申请提供的双向帧间预测方法可以由双向帧间预测装置、视频编解码装置、视频编解码器以及其它具有视频编解码功能的设备来执行。该双向帧间预测方法既可以发生在编码过程，也可以发生在解码过程，本申请实施例对此不作具体限定。
本申请实施例提供的双向帧间预测方法适用于视频传输系统。图3是本申请实施例提供的视频传输系统的结构示意图。如图3所示，该视频传输系统包括采集模块3001、 编码模块3002、发送模块3003、网络3004、接收模块3005、解码模块3006和渲染模块3007。
其中，视频传输系统中各个模块的具体作用如下：
采集模块3001包含摄像头或者摄像头组，用于采集视频图像，并对采集到的视频图像进行编码前的处理，将光信号转化为数字化的视频序列。
编码模块3002用于对视频序列进行编码，得到码流。
发送模块3003用于将编码得到的码流发送出去。
接收模块3005用于接收发送模块3003发送的码流。
网络3004用于将发送模块3003发送的码流传输到接收模块3005。
解码模块3006用于对接收模块3005接收的码流进行解码，重建视频序列。
渲染模块3007用于对解码模块3006解码得到的重建视频序列进行渲染，以提高视频的显示效果。
具体的，图3所示的视频传输系统中的编码模块3001和解码模块3006都可以执行本申请实施例的双向帧间预测方法。
从上述描述可知，本申请实施例提供的双向帧间预测方法可以由视频编解码装置执行。这样，本申请实施例提供的双向帧间预测方法也适用于由视频编解码装置组成的编解码系统。
下面结合图4A和图4B对编解码装置和由编解码装置组成的编解码系统进行详细的介绍。
图4A是本申请实施例的视频编解码装置的示意性图。如图4A所示，视频编解码装置40可以是专门用于对视频图像进行编码和/或解码的装置，也可以是具有视频编解码功能的电子设备，进一步地，该视频编解码装置40可以是无线通信系统的移动终端或者用户设备。
视频编解码装置40可以包括下列模块或者单元：控制器401、编解码器402、无线电接口403、天线404、智能卡405、读卡器406、小键盘407、存储器408、红外线端口409、显示器410。此外，除了图4A中所示的模块和单元之外，视频编解码装置40还可以包括麦克风或者任何适当的音频输入模块，该音频输入模块可以是数字或者模拟信号输入。相应的，视频编解码装置40还可以包括音频输出模块，该音频输出模块可以是耳机、扬声器或者模拟音频或者数字音频输出连接。视频编解码装置40也可以包括电池，该电池可以是太阳能电池、燃料电池等等。视频编解码装置40还可以包括用于与其它设备进行近程视线通信的红外线端口，该视频编解码装置40还可以采用包括任何适当的近程通信方式与其它设备进行通信，例如，蓝牙无线连接、USB/火线有线连接。
存储器408可以存储形式为图像的数据和音频的数据，也可以存储用于在控制器401上执行的指令。
视频编解码器40可以实现对音频和/或视频数据的编码和解码或者在控制器401的控制下实现对音频和/或视频数据的辅助编码和辅助解码。
智能卡405和读卡器406可以提供用户信息，也可以提供网络认证和授权用户的认证信息。智能卡405和读卡器406的具体实现形式可以是集成电路卡(Universal  Integrated Circuit Card，UICC)和UICC读取器。
无线电接口403可以生成无线通信信号，该无线通信信号可以是在进行蜂窝通信网络、无线通信系统或者无线局域网通信产生的通信信号。
天线404用于向其它装置(装置的数目可以为一个也可以为多个)发送在无线电接口电路403生成的射频信号，并且还可以用于从其它装置(装置的数目可以为一个也可以为多个)接收射频信号。
在本申请的一些实施例中，视频编解码装置40可以在传输和/或存储之前，接收另一设备发送的待处理的视频图像数据。在本申请的另一些实施例中，视频编解码装置40可以通过无线或者有线连接接收图像并对接收到的图像进行编码/解码。
图4B是本申请实施例的视频编解码系统50的示意性框图。
如图4B所示，视频编解码系统50包含源装置501及目的装置502。源装置501产生经过编码后的视频数据，源装置501也可以被称为视频编码装置或视频编码设备，目的装置502可以对源装置501产生的经过编码后的视频数据进行解码，目的装置502也可以被称为视频解码装置或视频解码设备。
源装置501和目的装置502的具体实现形式可以是如下设备中的任意一种：台式计算机、移动计算装置、笔记本(例如，膝上型)计算机、平板计算机、机顶盒、智能电话、手持机、电视、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机，或者其它类似的设备。
目的装置502可以经由信道503接收来自源装置501编码后的视频数据。信道503可包括能够将编码后的视频数据从源装置501移动到目的装置502的一个或多个媒体及/或装置。在一个实例中，信道503可以包括使源装置501能够实时地将编码后的视频数据直接发射到目的装置502的一个或多个通信媒体，在此实例中，源装置501可以根据通信标准(例如，无线通信协议)来调制编码后的视频数据，并且可以将调制后的视频数据发射到目的装置502。上述一个或多个通信媒体可以包含无线及/或有线通信媒体，例如射频(Radio Frequency，RF)频谱或一根或多根物理传输线。上述一个或多个通信媒体可以形成基于包的网络(例如，局域网、广域网或全球网络(例如，因特网))的部分。上述一个或多个通信媒体可以包含路由器、交换器、基站，或者实现从源装置501到目的装置502的通信的其它设备。
在另一实例中，信道503可包含存储由源装置501产生的编码后的视频数据的存储媒体。在此实例中，目的装置502可经由磁盘存取或卡存取来存取存储媒体。存储媒体可包含多种本地存取式数据存储媒体，例如蓝光光盘、高密度数字视频光盘(Digital Video Disc，DVD)、只读光盘(Compact Disc Read-Only Memory，CD-ROM)、快闪存储器，或用于存储经编码视频数据的其它合适数字存储媒体。
在另一实例中，信道503可包含文件服务器或存储由源装置501产生的编码后的视频数据的另一中间存储装置。在此实例中，目的装置502可经由流式传输或下载来存取存储于文件服务器或其它中间存储装置处的编码后的视频数据。文件服务器可以是能够存储编码后的视频数据且将所述编码后的视频数据发射到目的装置502的服务器类型。例如，文件服务器可以包含全球广域网(World Wide Web，Web)服务器(例如，用于网站)、文件传送协议(File Transfer Protocol，FTP)服务器、网络附加存 储(Network Attached Storage，NAS)装置以及本地磁盘驱动器。
目的装置502可经由标准数据连接(例如，因特网连接)来存取编码后的视频数据。数据连接的实例类型包含适合于存取存储于文件服务器上的编码后的视频数据的无线信道、有线连接(例如，缆线调制解调器等)，或两者的组合。编码后的视频数据从文件服务器的发射可为流式传输、下载传输或两者的组合。
本申请的双向帧间预测方法不限于无线应用场景，示例性的，本申请的双向帧间预测方法可以应用于支持以下应用等多种多媒体应用的视频编解码：空中电视广播、有线电视发射、卫星电视发射、流式传输视频发射(例如，经由因特网)、存储于数据存储媒体上的视频数据的编码、存储于数据存储媒体上的视频数据的解码，或其它应用。在一些实例中，视频编解码系统50可经配置以支持单向或双向视频发射，以支持例如视频流式传输、视频播放、视频广播及/或视频电话等应用。
在图4B中，源装置501包含视频源5011、视频编码器5012及输出接口5013。在一些实例中，输出接口5013可包含调制器/解调器(调制解调器)及/或发射器。视频源5011可包含视频俘获装置(例如，视频相机)、含有先前俘获的视频数据的视频存档、用以从视频内容提供者接收视频数据的视频输入接口，及/或用于产生视频数据的计算机图形系统，或上述视频数据源的组合。
视频编码器5012可编码来自视频源5011的视频数据。在一些实例中，源装置501经由输出接口5013将编码后的视频数据直接发射到目的装置502。编码后的视频数据还可存储于存储媒体或文件服务器上以供目的装置502稍后存取以用于解码及/或播放。
在图4B的实例中，目的装置502包含输入接口5023、视频解码器5022及显示装置5021。在一些实例中，输入接口5023包含接收器及/或调制解调器。输入接口5023可经由信道503接收编码后的视频数据。显示装置5021可与目的装置502整合或可在目的装置502外部。一般来说，显示装置5021显示解码后的视频数据。显示装置5021可包括多种显示装置，例如液晶显示器、等离子体显示器、有机发光二极管显示器或其它类型的显示装置。
视频编码器5012及视频解码器5022可根据视频压缩标准(例如，高效率视频编解码H.265标准)而操作，并且可以可遵照高效视频编码(High Efficiency Video Coding，HEVC)测试模型(HM)。
下面结合图3所示的视频传输系统、图4A示出的视频编解码装置以及图4B示出的由视频编解码装置组成的编解码系统对本申请提供的双向帧间预测方法进行详细描述。
本申请实施例提供的双向帧间预测方法可以由双向帧间预测装置执行，也可以由视频编解码装置执行，还可以由视频编解码器执行，还可以由其它具有视频编解码功能的设备执行，本申请实施例对此不作具体限定。
具体的，请参见图5，图5为本申请实施例提供的双向帧间预测方法的流程示意图。图5所示的双向帧间预测方法既可以发生在编码过程，也可以发生在解码过程。更具体的，图5所示的双向帧间预测方法可以发生在编解码时的帧间预测过程。
为了便于说明，图5中以双向帧间预测装置为执行主体为例进行说明。
如图5所示，该双向帧间预测方法包括：
S500、双向帧间预测装置根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块。
可选的，当前图像块的双向运动信息包括用于指示第一参考帧的第一参考帧索引、用于指示第二参考帧的第二参考帧索引、第一运动矢量以及第二运动矢量。
在这种情况下，双向帧间预测装置确定第一参考图像块和第二参考图像块的方法为：双向帧间预测装置获取第一参考帧索引、第二参考帧索引、第一运动矢量和第二运动矢量；双向帧间预测装置根据第一参考帧索引，从第一参考帧列表中获取第一参考帧，并根据第二参考帧索引，从第二参考帧列表中获取第二参考帧；双向帧间预测装置根据第一运动矢量和第一参考帧，确定第一参考图像块，并根据第二运动矢量和第二参考帧，确定第二参考图像块。
其中，第一参考帧与第二参考帧可以为同一参考帧，也可以为不同参考帧，本申请实施例对此不作具体限定。为了便于理解，本申请实施例后续内容均以第一参考帧与第二参考帧为不同参考帧为例进行说明。
本申请实施例中的第一参考帧索引和第二参考帧索引可以在码流中标识。例如：在HEVC标准中，语法元素ref_idx_l0和ref_idx_l1分别表示前向参考帧列表L0和后向参考帧列表L1中的参考帧索引。这样，双向帧间预测装置从码流中获取第一参考帧索引和第二参考帧索引。
可选的，双向帧间预测装置也可以从当前图像块的一个双向运动矢量预测值中，取前向运动矢量对应的参考帧索引作为第一参考索引，取后向运动矢量对应的参考帧索引作为第二参考帧索引。
其中，双向运动矢量预测值为当前图像块的运动矢量预测值候选者列表(Motion Vector Predictor candidate list，MVP candidate list)中的一个运动矢量预测值。当前图像块的运动矢量预测值候选者列表可包含单向运动矢量或双向运动矢量。
可选的，双向帧间预测装置还可以将当前图像块的前向运动矢量预测值的参考帧索引作为第一参考索引，并将当前图像块的后向运动矢量预测值的参考帧索引作为第一参考索引。
其中，前向运动矢量预测值为当前图像块的前向运动矢量预测值候选者列表中的一个运动矢量预测值，后向运动矢量预测值为当前图像块的后向运动矢量预测值候选者列表中的一个运动矢量预测值。当前图像块的前向运动矢量预测值候选者列表包含单向运动矢量，当前图像块的后向运动矢量预测值候选者列表包含单向运动矢量。
本申请实施例中的双向帧间预测装置还可以采用其他方式获取第一参考帧索引和第二参考帧索引，本申请实施例对此不作具体限定。
本申请实施例中的第一运动矢量和第二运动矢量均为单向运动矢量。第一运动矢量和第二运动矢量均可以由运动矢量预测值(Motion Vector Predictor，MVP)和运动矢量差分值(Motion Vector Difference，MVD)相加得到，也可以仅根据运动矢量预测值确定，还可以由解码端运动矢量搜索得到。本申请实施例对第一运动矢量和第二运动矢量的确定方式不作具体限定。
可选的，双向帧间预测装置还可以根据搜索模板在第一参考帧中确定第一参考图像块，根据搜索模板在第二参考帧中确定第二参考图像块。该方法在图7和图9所述 的实施例中详细描述。
S501、双向帧间预测装置获取当前模板、第一参考模板和第二参考模板。
当前模板包括当前图像块相邻、且已重建的像素区域，第一参考模板包括与第一参考图像块相邻、且已重建的像素区域，第二参考模板包括与第二参考图像块相邻、且已重建的像素区域。
本申请实施例中“当前模板与当前图像块的位置关系”、“第一参考模板与第一参考图像块的位置关系”以及“第二参考模板与第二参考图像块的位置关系”相同。
结合上述图像块的模板的定义可知，某一图像块的模板可包括该图像块上侧、左侧、右侧中至少一侧的相邻、已重建像素区域。
在一个示例中，如图6A所示，令T0为第一参考模板，T1为第二参考模板，Tc为当前模板，若当前模板Tc包括与当前图像块相邻、位于当前图像块左侧已重建的两列像素以及位于当前图像块上侧已重建的两行像素，则第一参考模板T0包括与第一参考图像块相邻、位于第一参考图像块左侧已重建的两列像素以及位于第一参考图像块上侧已重建的两行像素，第二参考模板T1包括与第二参考图像块相邻、位于第二参考图像块左侧已重建的两列像素以及位于第二参考图像块上侧已重建的两行像素。
在另一个示例中，如图6B所示，令T0为第一参考模板，T1为第二参考模板，Tc为当前模板，若当前模板Tc包括与当前图像块相邻、位于当前图像块左侧已重建的一列像素，则第一参考模板T0包括与第一参考图像块相邻、位于第一参考图像块左侧已重建的一列像素，第二参考模板T1包括与第二参考图像块相邻、位于第二参考图像块左侧已重建的一列像素。
此外，双向帧间预测装置可以在确定第一参考图像块和第二参考图像块之后获取当前模板，也可以在确定第一参考图像块和第二参考图像块之前获取当前模板，本申请实施例对此不作具体限定。
结合上述图像块的模板的定义可知，某一图像块的模板可包括该图像块上侧、左侧、右侧中至少一侧的相邻、已重建像素区域。
可选的，双向帧间预测装置可以根据位于某一图像块(当前图像块、第一参考图像块或第二参考图像块)周围、已经重建的区域，确定该图像块的模板。
示例性的，若当前图像块仅有上侧相邻区域已经重建，则当前模板包括当前图像块的上侧相邻区域；若当前图像块的上侧和左侧相邻区域均已重建，则当前模板包括当前图像块的上侧相邻区域和左侧相邻区域；若当前图像块的上侧和右侧相邻区域均已重建，则当前模板包括当前图像块的上侧相邻区域和右侧相邻区域。
可选的，双向帧间预测装置还可以根据某一图像块(当前图像块、第一参考图像块或第二参考图像块)的相邻区域的运动矢量和该图像块的运动矢量的相似性，确定该图像块的模板。
示例性的，双向帧间预测装置判断当前图像块的上侧相邻区域、左侧相邻区域、右侧相邻区域中至少一个区域的前向运动矢量是否与当前图像块的前向运动矢量相似，并确定前向运动矢量相似的区域属于当前模板；或者，双向帧间预测装置判断当前图像块的上侧相邻区域、左侧相邻区域、右侧相邻区域中至少一个区域的后向运动矢量是否与当前图像块的后向运动矢量相似，并确定后向运动矢量相似的区域属于当前模 板。
这里，双向帧间预测装置判断两个运动矢量相似的判断方法可以为：若两个运动矢量指向同一参考帧且二者之间的水平分量差值和竖直分量差值的绝对值之和小于预设阈值(例如4个、8个或12个像素)，则这两个运动矢量相似，否则为不相似。
本申请实施例中的双向帧间预测装置还可以采用其他方式获取某一图像块的模板，本申请实施例对此不作具体限定。
需要说明的是，在当前图像块、第一参考图像块或第二参考图像块的模板为空的情况下，双向帧间预测装置在执行S501后，可直接将第一参考图像块和第二参考图像块对应的权重值确定为1/2和1/2。
S502、双向帧间预测装置计算当前模板与第一参考模板之间的第一差异值，以及当前模板与第二参考模板之间的第二差异值。
为了便于描述，以当前模板为Tc、第一参考模板为T0、第二参考模板为T1、第一差异值为S0、第二差异值为S1为例进行说明。
可选的，双向帧间预测装置可以根据SAD确定差异值，即双向帧间预测装置根据Tc和T0对应像素的SAD确定S0，根据Tc和T1对应像素的SAD确定S1。S0越大，则表示当前模板与第一参考模板的相似程度越小；S1越大，则表示当前模板与第二参考模板的相似程度越小。
具体的，在双向帧间预测装置根据SAD确定差异值的情况下，双向帧间预测装置根据下述SAD计算公式(1)计算S0，根据下述SAD计算公式(2)计算S1。
Figure PCTCN2018119430-appb-000004
Figure PCTCN2018119430-appb-000005
上述公式(1)和公式(2)中，f c(x,y)为当前图像块中一个位置为(x,y)的像素(简称：像素(x,y))的亮度分量值，像素(x,y)与当前图像块的左上角的像素(0，0)的位置偏移为(x,y)，其中，x为水平方向偏移，y为竖直方向偏移；N1为当前图像块在水平方向的长度，N2为当前图像块在竖直方向的长度；f 0(x-dx0,y-dy0)为像素(x,y)根据运动矢量(dx0,dy0)得到的第一参考帧中对应位置的亮度分量值；f 1(x-dx1,y-dy1)为像素(x,y)根据运动矢量(dx1,dy1)得到的第二参考帧中对应位置的亮度分量值。
上述公式(1)和公式(2)中的差异值均为亮度分量值的SAD。可选的，差本申请实施例中的差异值还可以为亮度分量值的SAD和色度分量值的SAD的总和。
可选的，双向帧间预测装置还可以根据SATD确定差异值，即双向帧间预测装置根据Tc和T0对应像素的SATD确定S0，根据Tc和T1对应像素的SATD确定S1。S0越大，则表示当前模板与第一参考模板的相似程度越小；S1越大，则表示当前模板与第二参考模板的相似程度越小。
具体的，在双向帧间预测装置根据SATD确定差异值的情况下，双向帧间预测装置根据下述SATD计算公式(3)计算S0，根据下述SATD计算公式(4)计算S1。
Figure PCTCN2018119430-appb-000006
Figure PCTCN2018119430-appb-000007
上述公式(3)和公式(4)中，F c(μ,ν)＝H[f c(x,y)]，F 0(μ,ν)＝H[f 0(x-dx0,y-dy0)]，F 1(μ,ν)＝H[f 1(x-dx1,y-dy1)]，其中，H[x]表示对x进行变换，例如：哈达玛变换(Hadamard transform)，μ和ν为变换域的二维坐标值。公式(3)和公式(4)中的其他参数的描述可以参考上述公式(1)和公式(2)的描述，此处不再进行详细赘述。
可选的，双向帧间预测装置还可以根据MR-SAD确定差异值，即双向帧间预测装置根据Tc和T0对应像素的MR-SAD确定S0，根据Tc和T1对应像素的MR-SAD确定S1。S0越大，则表示当前模板与第一参考模板的相似程度越小；S1越大，则表示当前模板与第二参考模板的相似程度越小。
具体的，在双向帧间预测装置根据MR-SAD确定差异值的情况下，双向帧间预测装置根据下述MR-SAD计算公式(5)计算S0，根据下述MR-SAD计算公式(6)计算S1。
Figure PCTCN2018119430-appb-000008
Figure PCTCN2018119430-appb-000009
其中，DeltaC0为Tc和T0的亮度均值的差值，DeltaC1为Tc和T1的亮度均值的差值，即，
Figure PCTCN2018119430-appb-000010
Figure PCTCN2018119430-appb-000011
公式(5)和公式(6)中的其他参数的描述可以参考上述公式(1)和公式(2)的描述，此处不再进行详细赘述。
容易理解的是，本申请实施例中的双向帧间预测装置还可以根据其他准则确定差异值，这里不再一一举例说明。
S503、双向帧间预测装置确定用于表征第一差异值与第二差异值的差异程度的差距，根据该差距归属的预设范围，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值。
当第一差异值和第二差异值的差异程度较小时，可以确定第一参考图像块与当前图像块的相似程度接近于第二参考图像块与当前图像块的相似程度，因此，双向帧间预测装置可以对两个参考图像块均匀加权，即第一参考图像块对应的权重值等于第二参考图像块对应的权重值。当第一差异值与第二差异值的差异程度较大时，以第一差异值远小于第二差异值为例进行说明，可以确定第一参考图像块与当前图像块的相似程度高于第二参考图像块与当前图像块的相似程度，因此，双 向帧间预测装置为第一参考图像块分配较大的权重值，并为第二参考图像块分配较小的权重值，这样，可使得预测图像块在统计意义上更接近于当前图像块。
通常情况下，第一参考图像块对应的权重值与第二参考图像块的之和等于1。
具体的，双向帧间预测装置在计算出第一差异值和第二差异值之后，先确定用于表征第一差异值与第二差异值的差异程度的差距。
在一个示例中，双向帧间预测装置根据下述公式(7)或下述公式(8)确定用于表征两个参考图像块与当前图像块的差异值的差异程度的差距D，并根据该差距D为第一参考图像块和第二参考图像块分配权重值。
D＝(S0 n+k)/(S1 n+k)        (7)
D＝(S1 n+k)/(S0 n+k)        (8)
上述公式(7)和公式(8)中，参数n为第一预设常数，k为第二预设常数，例如：k＝0、0.5、1或3，n＝1/2、1或2。
两个差异值的差异程度越大(即上述公式(7)和公式(8)中D的数值越接近0或正无穷大)，权重分配越不均匀，即两个参考图像块对应的权重值差别越大。两个差异值的差异程度越小(即上述公式(7)和公式(8)中D的数值越接近1)，权重分配越均匀，即两个参考图像块对应的权重值差别越小。
在另一个示例中，双向帧间预测装置根据下述公式(9)或下述公式(10)确定用于表征两个参考图像块与当前图像块的差异值的差异程度的差距D，并根据该差距D为第一参考图像块和第二参考图像块分配权重值。
D＝S0 m-S1 m       (9)
D＝S1 m-S0 m       (10)
上述公式(9)和公式(10)中，参数m为第三预设常数，例如：m＝1或2。
两个差异值的差异程度越大(即上述公式(9)和公式(10)中D的数值越接近负无穷大或正无穷大)，权重分配越不均匀，即两个参考图像块对应的权重值差别越大。两个差异值的差异程度越小(即上述公式(9)和公式(10)中D的数值越接近于0)，权重分配越均匀，即两个参考图像块对应的权重值差别越小。
为减小实现复杂度，一种较简单权重分配方式为：根据上述差距D，在预设的多组权重值中，选择一组权重值，作为两个参考图像块对应的权重值。
双向帧间预测装置在确定出上述差距D后，根据该差距D归属的预设范围确定第一参考图像块对应的权重值和第二参考图像块对应的权重值。
可选的，双向帧间预测装置可根据差距D归属的预设范围和预设对应关系，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值；也可以根据上述D和预设公式，计算确定第一参考图像块对应的权重值和第二参考图像块对应的权重值。
现在对“双向帧间预测装置根据D和预设对应关系，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值”进行说明。
具体的，双向帧间预测装置从预设对应关系中查找与差距D归属的预设范围对应的权重值偏移量，这样，该双向帧间预测装置确定所述权重值偏移量与1/2之和为第一参考图像块对应的权重值，并确定1与第一参考图像块对应的权重值之差为第二参 考图像块对应的权重值。这里，预设对应关系包括多个预设差距范围以及每个预设差距范围对应的权重值偏移量，上述预设范围为所述多个预设差距范围中的一个。
在一个示例中，差距D使用上述公式(8)计算，权重值的精度为1/4(即权重值为1/4的整数倍)，预设对应关系如表1所示，记w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，均匀加权的权重值为Wa(通常Wa＝1/2)，第一参考图像块对应的权重值为w0＝Wa+Δw，第二参考图像块对应的权重值为w1＝1-w0＝1-Wa-Δw，w0与w1之和等于1。当“第二模板参考模板与当前模板的第二差异值”与“第一模板参考模板与当前模板的第一差异值”的比值大于阈值thred1(即D>thred1)时，Δw为正值(如表1中的1/4)，相应的，双向帧间预测装置确定出权重值{w0，w1}为{3/4，1/4}。当第二差异值与第一差异值的比值小于等于阈值thred1且大于等于阈值SmThd1(即D∈[SmThd1,thred)1]时，Δw为0，相应的，双向帧间预测装置确定出权重值{w0，w1}为{1/2，1/2}。当第二差异值与第一差异值的比值小于阈值SmThd1时，Δw为负值(如表1中的-1/4)，相应的，双向帧间预测装置确定出权重值{w0，w1}为{1/4，3/4}。这里，SmThd1的数值可以为0.5，thred1的数值可以为2。
表1
D <SmThd1 [SmThd1，thred1] >thred1
Δw -1/4 0/4 1/4
在另一个实例中，两个差异值之间的差异程度使用上述公式(8)计算，权重值的精度为1/8，预设对应关系如表2所示，记w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，均匀加权的权重值为Wa，第一参考图像块对应的权重值为w0＝Wa+Δw，第二参考图像块对应的权重值为w1＝Wa-Δw。
表2
Figure PCTCN2018119430-appb-000012
表2中的SmThd3、SmThd2、SmThd1、thred1、thred2和thred3用于表示不同的阈值，其中，thred1的数值可以为1.5，thred2的数值可以为2，thred3的数值可以为3，SmThd1的数值可以为2/3，SmThd2的数值可以为0.5，SmThd3的数值可以为1/3。
在另一个示例中，两个差异值之间的差异程度使用上述公式(8)计算，权重值的精度为1/16，预设对应关系如表3所示，记w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，均匀加权的权重值为Wa，第一参考图像块对应的权重值为w0＝Wa+Δw，第二参考图像块对应的权重值为w1＝Wa-Δw。
表3
D Δw
<SmThd7 -7/16
[SmThd7，SmThd6) -6/16
[SmThd6，SmThd5) -5/16
[SmThd5，SmThd4) -4/16
[SmThd4，SmThd3) -3/16
[SmThd3，SmThd2) -2/16
[SmThd2，SmThd1) -1/16
[SmThd1，thred1] 0/16
(thred1，thred2] 1/16
(thred2，thred3] 2/16
(thred3，thred4] 3/16
(thred4，thred5] 4/16
(thred5，thred6] 5/16
(thred6，thred7] 6/16
>Thred7 7/16
表3中的SmThd7、SmThd6、SmThd5、SmThd4、SmThd3、SmThd2、SmThd1、thred1、thred2、thred3、thred4、thred5、thred6和thred7用于表示不同的阈值，其中thred1的数值可以为1.25，thred2的数值可以为1.5，thred3的数值可以为1.75，thred4的数值可以为2，thred5的数值可以为2.25，thred6的数值可以为2.5，thred7的数值可以为2.75，SmThd[i]为thred[i]的倒数，i为1，2，…，7。
本申请实施例中，预设对应关系(如上述表1、表2、表3)中的阈值可利用指数哥伦布编码，被包括在序列头的码流中，或者预设对应关系中的阈值均为预设阈值。
实际应用中，由于除法的计算复杂度较大，上述示例还可以由如下实现方式来规避除法，减小复杂度：以上述表1为例，如果(S1 n+k)＞thred1·(S0 n+k)，则Δw为1/4，权重值{w0，w1}为{3/4，1/4}；如果(S1 n+k)＜SmThd1·(S0 n+k)，则Δw为-1/4，权重值{w0，w1}为{1/4，3/4}；否则，Δw为0，权重值{w0，w1}为{1/2，1/2}。
现在对“双向帧间预测装置根据D和预设公式，确定第一参考图像块对应的权重值和第二参考图像块对应的权重值”进行说明。
具体的，双向帧间预测装置根据预设公式，确定与差距D归属的预设范围对应的权重值偏移量，并确定第一参考图像块对应的权重值等于1/2与所述权重值偏移量之和，第二参考图像块对应的权重值为1减去第一参考图像块对应的权重值。
记Δw为第一差异值和第二差异值中较小值对应的参考图像块对应的权重值W min与均匀加权权重值Wa的差，即Δw＝W min-Wa，称为权重值偏移量，Δw通常为非负数。当第二差异值相比于第一差异值更大时，w0＝Wa+Δw，w1＝Wa-Δw；当第二差异值相比于第一差异值更小时，w0＝Wa-Δw，w1＝Wa+Δw。
其中，双向帧间预测装置根据下述公式(11)或公式(12)确定权重值偏移量Δw。
Figure PCTCN2018119430-appb-000013
Figure PCTCN2018119430-appb-000014
上述公式(11)和公式(12)中，floor(·)为向下取整函数，P为加权值的精度，如1/4、1/8或1/16等，o是第四预设常数，s表示线性关系的步长，其数值为预设常数，q表示对数关系的步长，其数值为第五预设常数，i为第六预设常数，D min为预设值，如D min＝1。
在一个示例中，若加权值的精度P的数值为1/8，o的数值为1，s的数值为2时，上述公式(11)可以表示为Δw＝(1/8)·floor(2·(D-o))。在这种情况下，Δw与D之间的对应关系如表4所示。
表4
D [1，1.5) [1.5，2) [2，2.5) ≥2.5
△w 0/8 1/8 2/8 3/8
若加权值的精度P的数值为1/8，o的数值为1，q和n的数值均为2时，上述公式(12)可以表示为Δw＝(1/8)·floor(log 2(4·(D-1))/log 2(2))。在这种情况下，Δw与D之间的对应关系如表5所示。
表5
D [1，1.5) [1.5，2) [2，3) ≥3
△w 0/8 1/8 2/8 3/8
结合上述表2，当表2中的thred1的数值为1.5，thred2的数值为2，thred3的数值为3，SmThd1的数值为2/3，SmThd2的数值为0.5，SmThd3的数值为1/3时，表2与表5的内容一致。
在一个示例中，若加权值的精度P的数值为1/6，o的数值为1.25，s的数值为2时，上述公式(11)可以表示为Δw＝(1/6)·floor(2(D-1))。在这种情况下，Δw与D之间的对应关系如表6所示。
表6
D [1.25，1.75) [1.75，2.25) ≥2.25
△w 0/6 1/6 2/6
若加权值的精度P的数值为1/6，o的数值为1，q的数值为
Figure PCTCN2018119430-appb-000015
n的数值为2时，上述公式(12)可以表示为：
Figure PCTCN2018119430-appb-000016
在这种情况下，Δw与D之间的对应关系如表7所示。
表7
Figure PCTCN2018119430-appb-000017
本申请实施例中，预设公式(如公式(11)、公式(12))中的相关参数(如P、o、s等)的数值可利用指数哥伦布编码，被包括在序列头的码流中，或者预设公式中的相关参数的数值均为预设数值。
可选的，为了进一步提高预测效率，如果min(S1,S0)＞Th1，则双向帧间预测装置确定权重值{w0，w1}为{1/2，1/2}。其中min(·)为取较小值操作，Th1为预设常数，例如：Th1＝10·NumP或Th1＝20·NumP，NumP为当前图像块包含的像素数目，即当前图像块的宽与当前图像块的高的乘积。
可选的，为了进一步提高预测效率，如果max(S1,S0)＞Th2，则双向帧间预测装置确定权重值{w0，w1}为{1/2，1/2}。其中max(·)为取较大值操作，Th2为预设常数，例如Th2＝20·NumP或Th2＝30·NumP。
S504、双向帧间预测装置根据第一参考图像块对应的权重值和第二参考图像块对应的权重值，对第一参考图像块和第二参考图像块进行加权，得到预测图像块。
双向帧间预测装置在确定出第一参考图像块对应的权重值和第二参考图像块对应的权重值后，根据第一参考图像块对应的权重值和第二参考图像块对应的权重值，对第一参考图像块的像素和第二参考图像块的像素进行加权，以得到当前图像块的预测图像块的像素。
可选的，双向帧间预测装置可根据下述公式(13)确定预测图像块P C。
P C＝w0·f 0+w1·f 1       (13)
上述公式(13)中，w0为第一参考图像块对应的权重值，w1为第二参考图像块对应的权重值，f 0为当前图像块根据第一运动矢量(dx0,dy0)得到的在第一参考帧中对应位置的第一参考图像块，f 1为当前图像块根据第二运动矢量(dx1,dy1)得到的在第二参考帧中对应位置的第二参考图像块。f 0和f 1中一组对应位置的像素按照w0和w1加权求和，得到P C中对应的像素，即P C(x,y)＝w0·f 0(x,y)+w1·f 1(x,y)。本申请实施例涉及到的像素可以仅包含亮度分量值，也可以包含亮度分量值和色度分量值。
需要说明的是，当第一运动矢量(dx0,dy0)和第二运动矢量(dx1,dy1)为亚像素精度的运动矢量时，第一参考图像块f 0和第二参考图像块f 1由空间位置的整像素插值得到。这里的插值方法可以为H.265标准中的亚像素插值方法，也可以为线性插值方法，还可以为其他插值方法，本申请实施例对此不作具体限定。
需要说明的是，上述自适应的权重值可以只应用于两个参考图像块的亮度分量的加权平均，色度分量的加权平均仍使用均匀加权方式。
可选的，双向帧间预测装置也可以只确定第一参考图像块中靠近第一参考模板的区域中的像素的权重值为w0，第二参考图像块中靠近第二参考模板的区域中的像素的权重值为w1，而第一参考图像块和第二参考图像块的其余区域中的像素的权重值仍然为1/2。对于第一参考图像块和第二参考图像块中对应位置的像素，双向帧间预测装置根据该像素的权重进行加权，得到预测图像块中对应的像素。
上述靠近模板的区域例如为符合max(x，y)≤k·min(M，H)的区域，其中，max(·)为取较大值操作，min(·)为取较小值操作，k为预设常数，如k＝1/2或1/4，M和H分别表示当前图像块的长和宽。当前图像块的长与参考图像块的长相同，当前图像块的宽与参考图像块的宽相同。
可选的，双向帧间预测装置还可以根据下述公式(14)确定预测图像块P C。
Figure PCTCN2018119430-appb-000018
上述公式(14)中，P C(x,y)为P C中位置为(x,y)的像素值，a＝max(x，y)，即a为x和y中的较大者，b为第七预设常数，例如b＝4或b＝c·min(M，H)，c为第八预设常数，c＝1/2、1/4、3/4或1。
上述图6A和图6B还简要的示出了本申请实施例中双向帧间预测装置按照图5所示的方法确定预测图像块的流程。
综上所述，双向帧间预测装置基于当前模板与第一参考模板的差异值以及当前模板与第二参考模板的差异值，确定用于表征两个差异值之间的差异程度的差距，并根据该差距归属的预设范围确定出了第一参考图像块对应的权重值和第二参考图像块对应的权重值，即双向帧间预测装置确定出了块级的权重值。
相比于现有技术，本申请实施例提供的双向帧间预测方法有效的提高了当前图像块与预测图像块之间的相似程度。
结合上面描述可知，本申请实施例中的双向帧间预测装置可以根据第一参考帧索引、第二参考帧索引、第一运动矢量以及第二运动矢量确定第一参考图像块和第二参考图像块，还可以根据搜索模板确定第一参考图像块和第二参考图像块。
在双向帧间预测装置根据搜索模板确定第一参考图像块和第二参考图像的场景中，该双向帧间预测装置需要先确定出搜索模板。双向帧间预测装置根据第一参考帧中的第一图像块以及第二参考帧中的第二图像块确定搜索模板。
可选的，双向帧间预测装置可以从预设的N对权重值中选取一对作为第一图像块对应的权重值和第二图像块对应的权重值，并按照该权重值对第一图像块和第二图像块进行加权，以确定搜索模板；也可以通过计算第一图像块的模板(即第一目标模板)和第二图像块的模板(即第二目标模板)分别与当前模板的差异值，并根据该差异值确定第一图像块对应的权重值和第二图像块对应的权重值，并按照该权重值对第一图像块的像素和第二图像块的像素进行加权，以确定搜索模板。
这里，先对双向帧间预测装置从预设的N对权重值中选取一对作为第一图像块对应的权重值和第二图像块对应的权重值，并按照该权重值对第一图像块的像素和第二图像块的像素进行加权，以确定搜索模板的像素，进而根据搜索模板确定第一参考图像块和第二参考图像块进行说明。
具体的，结合上述图5，如图7所示，本申请实施例中的S500包括S500a～S500h，S501包括S501a。
S500a、双向帧间预测装置获取用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量。
S500b、双向帧间预测装置根据第三参考帧索引和第三运动矢量，从第一参考帧中确定第一图像块，以及根据第四参考帧索引和第四运动矢量，从第二参考帧中确定第二图像块。
S500c、双向帧间预测装置获取当前模板、第一目标模板和第二目标模板。
第一目标模板包括第一图像块相邻、且已重建的像素区域，第二目标模板包括第二图像块相邻、且已重建的像素区域。
与第一参考模板和第二参考模板类似，本申请实施例中“当前模板与当前图像块的相对位置关系”、“第一目标模板与第一图像块的相对位置关系”以及“第二目标模板与第二图像块的相对位置关系”是相同的。
示例性的，如图8所示，Tp0是第一图像块的模板，即Tp0为第一目标模板；Tp1是第二图像块的模板，即Tp1为第二目标模板；Tc为当前图像块的模板，即Tc为当前模板。
S500d、双向帧间预测装置根据N(N≥2)对预设权重值，对第一目标模板的像素和第二目标模板的像素分别按照每一对预设权重值进行加权，得到N个合成空间模板。
示例性的，记第一图像块对应的权重值为wp0，第二图像块对应的权重值为wp1，若权重值的精度为1/6，则存在5对预设权重值，这5对预设权重值分别为{1/6，5/6}、{2/6，4/6}、{3/6，3/6}、{4/6，2/6}、{5/6，1/6}。双向帧间预测装置对第一目标模板的像素和第二目标模板的像素分别按照上述每一对预设权重值进行加权，得到5个合成空间模板。
可选的，双向帧间预测装置可采用S502示出的方法计算当前模板与第一目标模板的差异值，以及当前模板与第二目标模板的差异值，从而根据这两个差异值选择N对预设权重值。
示例性的，记第一图像块对应的权重值为wp0，第二图像块对应的权重值为wp1，若权重值的精度为1/6，则总共存在5对预设权重值，这5对预设权重值分别为{1/6，5/6}、{2/6，4/6}、{3/6，3/6}、{4/6，2/6}、{5/6，1/6}。若当前模板与第一目标模板的差异值小于当前模板与第二目标模板的差异值，则第一图像块对应的权重值应该大于等于第二图像块对应的权重值，因此，双向帧间预测装置只需根据下面三对预设权重值{3/6，3/6}、{4/6，2/6}、{5/6，1/6}对第一目标模板的像素和第二目标模板的像素进行加权，以生成三个不同的合成空间模板。
可选的，上述五对预设权重值还可以为{2/8，6/8}、{3/8，5/8}、{4/8，4/8}、{5/8，3/8}、{6/8，2/8}。
S500e、双向帧间预测装置计算当前模板与每一合成空间模板之间的差异值。
双向帧间预测装置可以根据SAD计算当前模板与每一合成空间模板之间的差异值，也可以为根据SATD计算当前模板与每一合成空间模板之间的差异值，还可以根据MR-SAD计算当前模板与每一合成空间模板之间的差异值，本申请实施例对此不作具体限定。
双向帧间预测装置计算当前模板与每一合成空间模板之间的差异值的具体方法可以参考上述S502的描述。
S500f、双向帧间预测装置确定差异值最小的合成空间模板，并将与该合成空间模板对应的一对预设权重值确定为第一图像块对应的权重值和第二图像块对应的权重值。
S500g、双向帧间预测装置根据第一图像块的权要值和第二图像块对应的权重值，对第一图像块的像素和第二图像块的像素进行加权，得到搜索模板。
具体的，双向帧间预测装置可根据下述公式(15)确定搜索模板S C。
S C＝wp0·P 0+wp1·P 1       (15)
上述公式(15)中，wp0为第一参考图像块对应的权重值，wp1为第二参考图像块对应的权重值，P 0为当前图像块根据第三运动矢量(dx2,dy2)得到的在第一参考帧中对应位置的第一图像块，P 1为当前图像块根据第四运动矢量(dx3,dy3)得到的在第二参考帧中对应位置的第二图像块。
需要说明的是，当(dx2,dy2)和(dx3,dy3)为亚像素精度的运动矢量时，第一图像块P 0(dx2,dy2)和第二图像块P 1(dx3,dy3)由空间位置的整像素插值得到。这里的插值方法可以为H.265标准中的亚像素插值方法，也可以为线性插值方法，还可以为其他插值方法，本申请实施例对此不作具体限定。
S500h、双向帧间预测装置根据搜索模板和第一参考帧，确定第一参考图像块，并根据搜索模板和第二参考帧，确定第二参考图像块。
具体的，双向帧间预测装置根据S500g中得到的搜索模板，在第一参考帧中以第一图像块为中心，预设大小的窗口区域内，选择一个与搜索模板的差异值最小的图像块，并将该图像块确定为第一参考图像块。可选的，双向帧间预测装置可以根据SAD、SATD或MR-SAD计算该差异值。
同理，双向帧间预测装置根据S500g中得到的搜索模板，在第二参考帧中以第二图像块为中心，预设大小的窗口区域内，选择一个与搜索模板的差异值最小的图像块，并将该图像块确定为第二参考图像块。
S501a、双向帧间预测装置获取第一参考模板和第二参考模板。
上述图8还简要的示出了本申请实施例中双向帧间预测装置按照图7示出的方法确定预测图像块的流程。
可以看出，双向帧间预测装置基于搜索模板确定第一参考图像块和第二参考图像块，这样，基于图5示出的实施例，本实施例提高了当前图像块与预测图像块之间的相似程度。
现对双向帧间预测装置过计算第一图像块的模板(即第一目标模板)和第二图像块的模板(即第二目标模板)分别与当前模板的差异值，并根据该差异值确定第一图像块对应的权重值和第二图像块对应的权重值，并按照该权重值对第一图像块的像素和第二图像块的像素进行加权，以确定搜索模板，进而根据搜索模板确定第一参考图像块和第二参考图像块进行说明。
具体的，结合上述图7，如图9所示，本申请实施例中的双向帧间预测装置在执行S500c之后，不再执行S500d～S500f，而是执行S500x～S500y，并在S500y之后，继续执行S500g。
S500x、双向帧间预测装置计算当前模板与第一目标模板之间的第三差异值，以及当前模板与第二目标模板之间的第四差异值。
可选的，双向帧间预测装置计算第三差异值以及第四差异值的方法可以参考上述S502，这里不再进行详细赘述。
S500y、双向帧间预测装置根据第三差异值与第四差异值，确定第一图像块对应的权重值和第二图像块对应的权重值。
当第三差异值与第四差异值之间的差值越大时，第一图像块对应的权重值与第二图像块对应的权重值之间的差值越大。若第三差异值远小于第四差异值，则双向帧间预测装置为第一图像块分配较大的权重值。相反的，若第四差异值远小于第三差异值，则双向帧间预测装置为第二图像块分配较大的权重值。差异值与权重值应该为非严格单调递增关系。
令第三差异值为S2、第四差异值为S3，w2为第一图像块对应的权重值，w3为第二图像块对应的权重值。
可选的，本申请实施例采用函数f(·)拟合权重值与差异值之间的关系，f(·)满足以下关系：f(S2，S3)+f(S3，S2)＝1，其中，f(x,y)表示x对应的图像块对应的权重值，即w2＝f(S2，S3)，w3＝f(S3，S2)。
示例性的，f(x,y)的表达式可以为下述任一公式：
Figure PCTCN2018119430-appb-000019
Figure PCTCN2018119430-appb-000020
Figure PCTCN2018119430-appb-000021
需要说明的是，S500y中双向帧间预测装置确定出的第一图像块对应的权重值和第二图像块对应的权重值可量化为以1/3、1/16、……、1/256为步长的精度，本申请实施例对此不作具体限定。
可选的，双向帧间预测装置确定第一图像块对应的权重值和第二图像块对应的权重值的方法可以参考上述S503的描述，此处不再进行详细赘述。
与图7所示的实施例类似，本实施例中的双向帧间预测装置也基于搜索模板确定第一参考图像块和第二参考图像块，这样，基于图5示出的实施例，本实施提高了当前图像块与预测图像块之间的相似程度。
本申请实施例提供一种双向帧间预测装置，该双向帧间预测装置用于执行以上双向帧间预测方法中的双向帧间预测装置所执行的步骤。本申请实施例提供的双向帧间预测装置可以包括相应步骤所对应的模块。
本申请实施例可以根据上述方法示例对双向帧间预测装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图10示出上述实施例中所涉及的双向帧间预测装置的一种可能的结构示意图。如图10所示，双向帧间预测装置10包括确定单元100、获取单元101、计算单元102以及加权单元103。
确定单元100用于支持该双向帧间预测装置10执行上述实施例中的S500、S503、S500f、S500h、S500y等，和/或用于本文所描述的技术的其它过程。
获取单元101用于支持该双向帧间预测装置10执行上述实施例中的S501、S500a、S500c等，和/或用于本文所描述的技术的其它过程。
计算单元102用于支持该双向帧间预测装置10执行上述实施例中的S502、S500e、S500x等，和/或用于本文所描述的技术的其它过程。
加权单元103用于支持该双向帧间预测装置10执行上述实施例中的S504、S500d、S500g等，和/或用于本文所描述的技术的其它过程。
其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
当然，本申请实施例提供的双向帧间预测装置10包括但不限于上述模块，例如：双向帧间预测装置还可以包括存储单元104。
存储单元104可以用于存储该双向帧间预测装置10的程序代码和数据。
在采用集成的单元的情况下，本申请实施例提供的双向帧间预测装置的结构示意图如图11所示。在图11中，该双向帧间预测装置11包括：处理模块110和通信模块111。处理模块110用于对双向帧间预测装置11的动作进行控制管理，例如，执行上述确定单元100、计算单元102以及加权单元103执行的步骤，和/或用于执行本文所描述的技术的其它过程。通信模块111用于支持双向帧间预测装置11与其他设备之间的交互，例如，执行上述获取单元101执行的步骤。如图11所示，双向帧间预测装置11还可以包括存储模块112，存储模块112用于存储双向帧间预测装置11的程序代码和数据，例如存储上述存储单元104所保存的内容。
其中，处理模块110可以是处理器或控制器，例如可以是中央处理器(Central Processing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，专用集成电路(Application-Specific Integrated Circuit，ASIC)，现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块111可以是收发器、RF电路或通信接口等。存储模块112可以是存储器。
其中，上述方法实施例涉及的各场景的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
上述双向帧间预测装置10和双向帧间预测装置11均可执行上述图5、图7和图9中任一附图所示的双向帧间预测方法，双向帧间预测装置10和双向帧间预测装置11具体可以是视频编码装置、视频解码装置或者其他具有视频编解码功能的设备。双向帧间预测装置10和双向帧间预测装置11既可以用于在编码过程中进行图像预测，也可以用于在解码过程中进行图像预测。
本申请还提供一种终端，该终端包括：一个或多个处理器、存储器、通信接口。该存储器、通信接口与一个或多个处理器耦合；存储器用于存储计算机程序代码，计算机程序代码包括指令，当一个或多个处理器执行指令时，终端执行本申请实施例的双向帧间预测方法。
这里的终端可以是视频显示设备，智能手机，便携式电脑以及其它可以处理视频 或者播放视频的设备。
本申请还提供一种视频编码器，包括非易失性存储介质，以及中央处理器，所述非易失性存储介质存储有可执行程序，所述中央处理器与所述非易失性存储介质连接，并执行所述可执行程序以实现本申请实施例的双向帧间预测方法。
本申请还提供一种视频解码器，包括非易失性存储介质，以及中央处理器，所述非易失性存储介质存储有可执行程序，所述中央处理器与所述非易失性存储介质连接，并执行所述可执行程序以实现本申请实施例的双向帧间预测方法。
本申请还提供一种解码器，所述解码器包括本申请实施例中的双向帧间预测装置(双向帧间预测装置10或双向帧间预测装置11)以及重建模块，其中，所述重建模块用于根据所述双向帧间预测装置得到的预测图像块的像素确定当前图像块的重建像素值。
本申请还提供一种编码器，所述编码器包括本申请实施例中的双向帧间预测装置(双向帧间预测装置10或双向帧间预测装置11)以及重建模块，其中，所述重建模块用于根据所述双向帧间预测装置得到的预测图像块的像素值确定当前图像块的重建像素值。
本申请另一实施例还提供一种计算机可读存储介质，该计算机可读存储介质包括一个或多个程序代码，该一个或多个程序包括指令，当终端中的处理器在执行该程序代码时，该终端执行如图5、图7和图9中任一附图所示的双向帧间预测方法。
在本申请的另一实施例中，还提供一种计算机程序产品，该计算机程序产品包括计算机执行指令，该计算机执行指令存储在计算机可读存储介质中；终端的至少一个处理器可以从计算机可读存储介质读取该计算机执行指令，至少一个处理器执行该计算机执行指令使得终端实施执行图5、图7和图9中任一附图所示的双向帧间预测方法中的双向帧间预测装置的步骤。
在上述实施例中，可以全部或部分的通过软件，硬件，固件或者其任意组合来实现。当使用软件程序实现时，可以全部或部分地以计算机程序产品的形式出现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。
所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。该可用介质可以是磁性介质，(例如，软盘，硬盘、磁带)、光介质(例如，DVD)或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模 块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何在本申请揭露的技术范围内的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (25)

  1. 一种双向帧间预测方法，其特征在于，包括：
    根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块；
    获取当前模板、第一参考模板和第二参考模板，所述当前模板包括与当前图像块相邻、且已重建的像素区域，所述第一参考模板包括与所述第一参考图像块相邻、且已重建的像素区域，所述第二参考模板包括与所述第二参考图像块相邻、且已重建的像素区域；所述当前模板与所述当前图像块的位置关系、所述第一参考模板与所述第一参考图像块的位置关系、以及所述第二参考模板与所述第二参考图像块的位置关系均相同；
    计算所述当前模板与所述第一参考模板之间的第一差异值，以及所述当前模板与所述第二参考模板之间的第二差异值；
    确定差距，所述差距用于表征所述第一差异值与所述第二差异值的差异程度；
    根据所述差距归属的预设范围，确定所述第一参考图像块对应的权重值和所述第二参考图像块对应的权重值；所述差距归属不同预设范围时，所述第一参考图像块对应的权重值不同，所述第二参考图像块对应的权重值不同；
    根据所述第一参考图像块对应的权重值和所述第二参考图像块对应的权重值，对所述第一参考图像块的像素和所述第二参考图像块的像素进行加权，得到预测图像块的像素。
  2. 根据权利要求1所述的双向帧间预测方法，其特征在于，所述确定差距，具体包括：
    根据下述公式确定所述差距：
    D＝(S1 n+k)/(S0 n+k)
    其中，D表示所述差距，S1表示所述第二差异值，S0表示所述第一差异值，n为第一预设常数，k为第二预设常数。
  3. 根据权利要求1所述的双向帧间预测方法，其特征在于，所述确定差距，具体包括：
    根据下述公式确定所述差距：
    D＝S1 m-S0 m
    其中，D表示所述差距，S1表示所述第二差异值，S0表示所述第一差异值，m为第三预设常数。
  4. 根据权利要求1-3中任意一项所述的双向帧间预测方法，其特征在于，所述根据所述差距归属的预设范围，确定所述第一参考图像块对应的权重值和所述第二参考图像块对应的权重值，具体包括：
    从预设对应关系中查找与所述预设范围对应的权重值偏移量；所述预设对应关系包括多个预设差距范围以及每个所述预设差距范围对应的权重值偏移量，所述预设范围为所述多个预设差距范围中的一个；
    根据第一预设公式和所述权重值偏移量，确定所述第一参考图像块对应的权重值，并根据第二预设公式以及所述第一参考图像块对应的权重值，确定所述第二参考图像块对应的权重值；
    其中，所述第一预设公式为：w0＝1/2+Δw；所述第二预设公式为：w1＝1-w0；
    w0为所述第一参考图像块对应的权重值，Δw为所述权重值偏移量，w1为所述第二参考图像块对应的权重值。
  5. 根据权利要求1-3中任意一项所述的双向帧间预测方法，其特征在于，所述根据所述差距归属的预设范围，确定所述第一参考图像块对应的权重值和所述第二参考图像块对应的权重值，具体包括：
    根据第三预设公式，确定与所述预设范围对应的权重值偏移量；
    根据第一预设公式和所述权重值偏移量，确定所述第一参考图像块对应的权重值，并根据第二预设公式和所述第一参考图像块对应的权重值，确定所述第二参考图像块对应的权重值；
    其中，所述第一预设公式为：w0＝1/2+Δw；所述第二预设公式为：w1＝1-w0；所述第三预设公式为：
    Figure PCTCN2018119430-appb-100001
    w0为所述第一参考图像块对应的权重值，w1为所述第二参考图像块对应的权重值，Δw表示所述权重值偏移量，floor(·)为向下取整函数，P表示权重值的精度，o是第四预设常数，q为第五预设常数，i为第六预设常数，D min表示所述差距D的最小值。
  6. 根据权利要求1-5中任意一项所述的双向帧间预测方法，其特征在于，所述当前图像块的双向运动信息包括用于指示第一参考帧的第一参考帧索引、用于指示第二参考帧的第二参考帧索引、第一运动矢量以及第二运动矢量；
    所述根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块，具体包括：
    获取所述第一参考帧索引、所述第二参考帧索引、所述第一运动矢量和所述第二运动矢量；
    根据所述第一参考帧索引，从第一参考帧列表中获取所述第一参考帧，并根据所述第二参考帧索引，从第二参考帧列表中获取所述第二参考帧；
    根据所述第一运动矢量和所述第一参考帧，确定第一参考图像块，并根据所述第二运动矢量和所述第二参考帧，确定第二参考图像块。
  7. 根据权利要求1-5中任意一项所述的双向帧间预测方法，其特征在于，所述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量；
    所述根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块，具体包括：
    获取所述第三参考帧索引、所述第四参考帧索引、所述第三运动矢量和所述第四运动矢量；
    根据所述第三参考帧索引和所述第三运动矢量，从所述第一参考帧中确定第一图像块，以及根据所述第四参考帧索引和所述第四运动矢量，从所述第二参考帧中确定第二图像块；
    获取第一目标模板和第二目标模板，所述第一目标模板包括所述第一图像块相邻、 且已重建的像素区域，所述第二目标模板包括所述第二图像块相邻、且已重建的像素区域；
    根据所述第一目标模板和所述第二目标模板分别与所述当前模板之间的差异值，确定所述第一图像块对应的权重值和所述第二图像块对应的权重值；
    根据所述第一图像块的权要值和所述第二图像块对应的权重值，对所述第一图像块的像素和所述第二图像块的像素进行加权，得到搜索模板；
    根据所述搜索模板，从所述第一参考帧中确定所述第一参考图像块，并从所述第二参考帧中确定所述第二参考图像块。
  8. 根据权利要求7所述的双向帧间预测方法，其特征在于，所述根据所述第一目标模板和所述第二目标模板分别与所述当前模板之间的差异值，确定所述第一图像块对应的权重值和所述第二图像块对应的权重值，具体包括：
    计算所述当前模板与所述第一目标模板之间的第三差异值，以及所述当前模板与所述第二目标模板之间的第四差异值；
    根据所述第三差异值与所述第四差异值，确定所述第一图像块对应的权重值和所述第二图像块对应的权重值。
  9. 根据权利要求7所述的双向帧间预测方法，其特征在于，所述根据所述第一目标模板和所述第二目标模板分别与所述当前模板之间的差异值，确定所述第一图像块对应的权重值和所述第二图像块对应的权重值，具体包括：
    根据N对预设权重值，对所述第一目标模板的像素和所述第二目标模板的像素分别按照每一对预设权重值进行加权，得到N个合成空间模板，N≥2；
    计算所述当前模板与每一合成空间模板之间的差异值；
    确定差异值最小的合成空间模板，根据与该合成空间模板对应的一对预设权重值确定所述第一图像块对应的权重值和所述第二图像块对应的权重值。
  10. 一种双向帧间预测装置，其特征在于，包括：
    确定单元，用于根据当前图像块的双向运动信息，确定第一参考图像块和第二参考图像块；
    获取单元，用于获取当前模板、第一参考模板和第二参考模板，所述当前模板包括与所述确定单元确定的当前图像块相邻、且已重建的像素区域，所述第一参考模板包括与所述确定单元确定的所述第一参考图像块相邻、且已重建的像素区域，所述第二参考模板包括与所述确定单元确定的所述第二参考图像块相邻、且已重建的像素区域；所述当前模板与所述当前图像块的位置关系、所述第一参考模板与所述第一参考图像块的位置关系、以及所述第二参考模板与所述第二参考图像块的位置关系均相同；
    计算单元，用于计算上述获取单元获取到的所述当前模板与所述第一参考模板之间的第一差异值，以及所述当前模板与所述第二参考模板之间的第二差异值；
    所述确定单元，还用于确定差距，所述差距用于表征所述第一差异值与所述第二差异值的差异程度，以及根据所述差距归属的预设范围，确定所述第一参考图像块对应的权重值和所述第二参考图像块对应的权重值；所述差距归属不同预设范围时，所述第一参考图像块对应的权重值不同，所述第二参考图像块对应的权重值不同；
    加权单元，用于根据所述确定单元确定出的所述第一参考图像块对应的权重值和 所述第二参考图像块对应的权重值，对所述第一参考图像块的像素和所述第二参考图像块的像素进行加权，得到预测图像块的像素。
  11. 根据权利要求10所述的双向帧间预测装置，其特征在于，所述确定单元具体用于：
    根据下述公式确定所述差距：
    D＝(S1 n+k)/(S0 n+k)
    其中，D表示所述差距，S1表示所述第二差异值，S0表示所述第一差异值，n为第一预设常数，k为第二预设常数。
  12. 根据权利要求10所述的双向帧间预测装置，其特征在于，所述确定单元具体用于：
    根据下述公式确定所述差距：
    D＝S1 m-S0 m
    其中，D表示所述差距，S1表示所述第二差异值，S0表示所述第一差异值，m为第三预设常数。
  13. 根据权利要求10-12中任意一项所述的双向帧间预测装置，其特征在于，所述确定单元具体用于：
    从预设对应关系中查找与所述预设范围对应的权重值偏移量；所述预设对应关系包括多个预设差距范围以及每个所述预设差距范围对应的权重值偏移量，所述预设范围为所述多个预设差距范围中的一个；
    根据第一预设公式和所述权重值偏移量，确定所述第一参考图像块对应的权重值，并根据第二预设公式以及所述第一参考图像块对应的权重值，确定所述第二参考图像块对应的权重值；
    其中，所述第一预设公式为：w0＝1/2+Δw；所述第二预设公式为：w1＝1-w0；
    w0为所述第一参考图像块对应的权重值，Δw为所述权重值偏移量，w1为所述第二参考图像块对应的权重值。
  14. 根据权利要求10-12中任意一项所述的双向帧间预测装置，其特征在于，所述确定单元具体用于：
    根据第三预设公式，确定与所述预设范围对应的权重值偏移量；
    根据第一预设公式和所述权重值偏移量，确定所述第一参考图像块对应的权重值，并根据第二预设公式和所述第一参考图像块对应的权重值，确定所述第二参考图像块对应的权重值；
    其中，所述第一预设公式为：w0＝1/2+Δw；所述第二预设公式为：w1＝1-w0；所述第三预设公式为：
    Figure PCTCN2018119430-appb-100002
    w0为所述第一参考图像块对应的权重值，w1为所述第二参考图像块对应的权重值，Δw表示所述权重值偏移量，floor(·)为向下取整函数，P表示权重值的精度，o是第四预设常数，q为第五预设常数，i为第六预设常数，D min表示所述差距D的最小值。
  15. 根据权利要求10-14中任意一项所述的双向帧间预测装置，其特征在于，所 述当前图像块的双向运动信息包括用于指示第一参考帧的第一参考帧索引、用于指示第二参考帧的第二参考帧索引、第一运动矢量以及第二运动矢量；
    所述获取单元，还用于获取所述第一参考帧索引、所述第二参考帧索引、所述第一运动矢量和所述第二运动矢量，以及根据所述第一参考帧索引，从第一参考帧列表中获取所述第一参考帧，并根据所述第二参考帧索引，从第二参考帧列表中获取所述第二参考帧；
    所述确定单元，具体用于根据所述获取单元获取到的所述第一运动矢量和所述第一参考帧，确定第一参考图像块，并根据所述第二运动矢量和所述第二参考帧，确定第二参考图像块。
  16. 根据权利要求10-14中任意一项所述的双向帧间预测装置，其特征在于，所述当前图像块的双向运动信息包括用于指示第一参考帧的第三参考帧索引、用于指示第二参考帧的第四参考帧索引、第三运动矢量和第四运动矢量；
    所述获取单元，还用于获取所述第三参考帧索引、所述第四参考帧索引、所述第三运动矢量和所述第四运动矢量，以及根据所述第三参考帧索引和所述第三运动矢量，从所述第一参考帧中确定第一图像块，以及根据所述第四参考帧索引和所述第四运动矢量，从所述第二参考帧中确定第二图像块，以及获取第一目标模板和第二目标模板，所述第一目标模板包括所述第一图像块相邻、且已重建的像素区域，所述第二目标模板包括所述第二图像块相邻、且已重建的像素区域；
    所述确定单元，具体用于根据所述获取单元获取到的所述第一目标模板和所述第二目标模板分别与所述当前模板之间的差异值，确定所述第一图像块对应的权重值和所述第二图像块对应的权重值；
    所述加权单元，还用于根据所述确定单元确定出的所述第一图像块的权要值和所述第二图像块对应的权重值，对所述第一图像块的像素和所述第二图像块的像素进行加权，得到搜索模板；
    所述确定单元，具体用于根据所述加权单元得到的所述搜索模板，从所述第一参考帧中确定所述第一参考图像块，并从所述第二参考帧中确定所述第二参考图像块。
  17. 根据权利要求16所述的双向帧间预测装置，其特征在于，
    所述计算单元，还用于计算所述当前模板与所述第一目标模板之间的第三差异值，以及所述当前模板与所述第二目标模板之间的第四差异值；
    所述确定单元，具体用于根据所述计算单元计算出的所述第三差异值与所述第四差异值，确定所述第一图像块对应的权重值和所述第二图像块对应的权重值。
  18. 根据权利要求16所述的双向帧间预测装置，其特征在于，
    所述加权单元，还用于根据N对预设权重值，对所述第一目标模板的像素和所述第二目标模板的像素分别按照每一对预设权重值进行加权，得到N个合成空间模板，N≥2；
    所述计算单元，还用于计算所述当前模板与所述加权单元得到的每一合成空间模板之间的差异值；
    所述确定单元，具体用于确定所述计算单元计算出的差异值最小的合成空间模板，根据与该合成空间模板对应的一对预设权重值确定所述第一图像块对应的权重值和所 述第二图像块对应的权重值。
  19. 一种终端，其特征在于，所述终端包括：一个或多个处理器、存储器和通信接口；
    所述存储器、所述通信接口与所述一个或多个处理器连接；所述终端通过所述通信接口与其他设备通信，所述存储器用于存储计算机程序代码，所述计算机程序代码包括指令，当所述一个或多个处理器执行所述指令时，所述终端执行如权利要求1-9中任意一项所述的双向帧间预测方法。
  20. 一种包含指令的计算机程序产品，其特征在于，当所述计算机程序产品在终端上运行时，使得所述终端执行如权利要求1-9中任意一项所述的双向帧间预测方法。
  21. 一种计算机可读存储介质，包括指令，其特征在于，当所述指令在终端上运行时，使得所述终端执行如权利要求1-9中任意一项所述的双向帧间预测方法。
  22. 一种视频编码器，包括非易失性存储介质以及中央处理器，其特征在于，所述非易失性存储介质存储有可执行程序，所述中央处理器与所述非易失性存储介质连接，当所述中央处理器执行所述可执行程序时，所述视频编码器执行如权利要求1-9中任意一项所述的双向帧间预测方法。
  23. 一种视频解码器，包括非易失性存储介质以及中央处理器，其特征在于，所述非易失性存储介质存储有可执行程序，所述中央处理器与所述非易失性存储介质连接，当所述中央处理器执行所述可执行程序时，所述视频解码器执行如权利要求1-9中任意一项所述的双向帧间预测方法。
  24. 一种编码器，其特征在于，所述编码器包括如权利要求10-18中任意一项所述的双向帧间预测装置以及重建模块，其中，所述重建模块用于根据所述双向帧间预测装置得到的预测图像块的像素确定当前图像块的重建像素值。
  25. 一种解码器，其特征在于，所述解码器包括如权利要求10-18中任意一项所述的双向帧间预测装置以及重建模块，其中，所述重建模块用于根据所述双向帧间预测装置得到的预测图像块的像素确定当前图像块的重建像素值。
PCT/CN2018/119430 2018-02-06 2018-12-05 一种双向帧间预测方法及装置 WO2019153861A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810118179.9A CN110121073B (zh) 2018-02-06 2018-02-06 一种双向帧间预测方法及装置
CN201810118179.9 2018-02-06

Publications (1)

Publication Number Publication Date
WO2019153861A1 true WO2019153861A1 (zh) 2019-08-15

Family

ID=67519914

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/119430 WO2019153861A1 (zh) 2018-02-06 2018-12-05 一种双向帧间预测方法及装置

Country Status (2)

Country Link
CN (1) CN110121073B (zh)
WO (1) WO2019153861A1 (zh)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111050177B (zh) * 2018-10-15 2021-07-20 腾讯科技（深圳）有限公司 视频编码、视频解码方法、装置、计算机设备和存储介质
CN112543323B (zh) * 2019-09-23 2022-05-31 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
JP7436646B2 (ja) * 2019-10-10 2024-02-21 華為技術有限公司 ピクチャヘッダのシグナリングを簡略化するためのエンコーダ、デコーダ及び対応する方法
CN111104930B (zh) * 2019-12-31 2023-07-11 腾讯科技（深圳）有限公司 视频处理方法、装置、电子设备及存储介质
CN113452997B (zh) * 2020-03-25 2022-07-29 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
CN113810686B (zh) * 2020-06-01 2023-02-24 杭州海康威视数字技术股份有限公司 一种编解码方法、装置及其设备
CN116320428A (zh) * 2021-12-20 2023-06-23 维沃移动通信有限公司 帧间预测方法及终端

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101931803A (zh) * 2009-06-26 2010-12-29 华为技术有限公司 视频图像运动信息获取方法、装置及设备、模板构造方法
US9438910B1 (en) * 2014-03-11 2016-09-06 Google Inc. Affine motion prediction in video coding
CN106105196A (zh) * 2014-04-28 2016-11-09 松下电器（美国）知识产权公司 编码方法、解码方法、编码装置以及解码装置
WO2017036417A1 (en) * 2015-09-06 2017-03-09 Mediatek Inc. Method and apparatus of adaptive inter prediction in video coding

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100703770B1 (ko) * 2005-03-25 2007-04-06 삼성전자주식회사 가중 예측을 이용한 비디오 코딩 및 디코딩 방법, 이를위한 장치
CN102160381A (zh) * 2008-09-24 2011-08-17 索尼公司 图像处理设备和方法
CN102215407B (zh) * 2010-04-09 2013-07-10 华为技术有限公司 双向预测块的获取方法和装置以及图像解码方法
CN111193930B (zh) * 2013-12-16 2021-11-30 浙江大学 一种前向双假设编码图像块的编解码方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101931803A (zh) * 2009-06-26 2010-12-29 华为技术有限公司 视频图像运动信息获取方法、装置及设备、模板构造方法
US9438910B1 (en) * 2014-03-11 2016-09-06 Google Inc. Affine motion prediction in video coding
CN106105196A (zh) * 2014-04-28 2016-11-09 松下电器（美国）知识产权公司 编码方法、解码方法、编码装置以及解码装置
WO2017036417A1 (en) * 2015-09-06 2017-03-09 Mediatek Inc. Method and apparatus of adaptive inter prediction in video coding

Also Published As

Publication number Publication date
CN110121073A (zh) 2019-08-13
CN110121073B (zh) 2021-07-09

Similar Documents

Publication Publication Date Title
WO2019153861A1 (zh) 一种双向帧间预测方法及装置
US20200145689A1 (en) Picture prediction method and picture prediction apparatus
JP7279154B2 (ja) アフィン動きモデルに基づく動きベクトル予測方法および装置
CN114501010B (zh) 图像编码方法、图像解码方法及相关装置
WO2017129023A1 (zh) 解码方法、编码方法、解码设备和编码设备
WO2019184639A1 (zh) 一种双向帧间预测方法及装置
US11924438B2 (en) Picture reconstruction method and apparatus
WO2019128716A1 (zh) 图像的预测方法、装置及编解码器
WO2020006969A1 (zh) 运动矢量预测方法以及相关装置
WO2019184556A1 (zh) 一种双向帧间预测方法及装置
CN113383550A (zh) 光流修正的提前终止
WO2020048205A1 (zh) 应用于双向帧间预测中的参考帧获取方法及装置
WO2019183906A1 (zh) 帧间预测的方法和装置
KR102407912B1 (ko) 양방향 인트라 예측 시그널링
WO2020143585A1 (zh) 视频编码器、视频解码器及相应方法
CN110876065A (zh) 候选运动信息列表的构建方法、帧间预测方法及装置
US11516470B2 (en) Video coder and corresponding method
US11323706B2 (en) Method and apparatus for aspect-ratio dependent filtering for intra-prediction
WO2019233423A1 (zh) 获取运动矢量的方法和装置
JP2023100767A (ja) ビデオシーケンスのためのピクチャエンコーディング及びデコーディング方法及び装置
CN111656786B (zh) 候选运动信息列表的构建方法、帧间预测方法及装置
WO2020181476A1 (zh) 视频图像预测方法及装置
WO2022111349A1 (zh) 图像处理方法、设备、存储介质及计算机程序产品
CN110971899B (zh) 一种确定运动信息的方法、帧间预测方法及装置
US11706444B2 (en) Inter prediction method and apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18905087

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18905087

Country of ref document: EP

Kind code of ref document: A1