WO2016065873A1 - 图像预测方法及相关装置 - Google Patents

图像预测方法及相关装置 Download PDF

Info

Publication number
WO2016065873A1
WO2016065873A1 PCT/CN2015/077295 CN2015077295W WO2016065873A1 WO 2016065873 A1 WO2016065873 A1 WO 2016065873A1 CN 2015077295 W CN2015077295 W CN 2015077295W WO 2016065873 A1 WO2016065873 A1 WO 2016065873A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion information
information unit
image block
pixel
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/077295
Other languages
English (en)
French (fr)
Chinese (zh)
Inventor
陈焕浜
林四新
梁凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to KR1020207002096A priority Critical patent/KR102248254B1/ko
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to KR1020217012948A priority patent/KR102334151B1/ko
Priority to KR1020177007312A priority patent/KR102013771B1/ko
Priority to BR112017007399-4A priority patent/BR112017007399B1/pt
Priority to EP15855692.8A priority patent/EP3177015B1/en
Priority to EP24154875.9A priority patent/EP4387229A3/en
Priority to KR1020197024188A priority patent/KR102070982B1/ko
Priority to JP2017517269A priority patent/JP6490203B2/ja
Publication of WO2016065873A1 publication Critical patent/WO2016065873A1/zh
Priority to US15/463,850 priority patent/US10440380B2/en
Anticipated expiration legal-status Critical
Priority to US16/431,298 priority patent/US10623763B2/en
Priority to US16/845,161 priority patent/US11172217B2/en
Priority to US17/511,269 priority patent/US11968386B2/en
Priority to US18/628,367 priority patent/US12382081B2/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/537—Motion estimation other than block-based
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to an image prediction method and related apparatus.
  • the basic principle of video coding compression is to use the correlation between airspace, time domain and codewords to remove redundancy as much as possible.
  • the current popular practice is to use a block-based hybrid video coding framework to implement video coding compression through prediction (including intra prediction and inter prediction), transform, quantization, and entropy coding.
  • This coding framework shows a strong vitality, and HEVC still uses this block-based hybrid video coding framework.
  • motion estimation/motion compensation is a key technique that affects encoding/decoding performance.
  • various various video encoding/decoding schemes existing, it is assumed that the motion of the object always satisfies the translational motion, and the respective parts of the entire object have the same motion.
  • the existing motion estimation/motion compensation algorithms are basically block motion compensation algorithms based on the translational motion model.
  • irregular movements such as scaling, rotation, and parabolic motion are ubiquitous.
  • video coding experts realized the universality of irregular motion and hoped to improve video by introducing irregular motion models (such as affine motion models, rotational motion models, and scaled motion models). Coding efficiency, but the computational complexity of existing image prediction based on non-translational motion models is usually very high.
  • Embodiments of the present invention provide an image prediction method and related apparatus, so as to reduce computational complexity of image prediction based on a non-translational motion model.
  • a first aspect of the present invention provides an image prediction method, including:
  • Determining K1 pixel samples in the image block x determining a candidate motion information unit set corresponding to each of the K1 pixel samples; wherein the candidate motion information unit set corresponding to each pixel sample Include at least one motion information unit of the candidate, wherein the K1 is an integer greater than or equal to 2;
  • Each of the motion information units i is selected from at least a portion of the motion information unit of the candidate motion information unit set corresponding to the different pixel samples of the K1 pixel samples, where
  • the motion information unit includes a motion vector in which the prediction direction is forward and/or a motion vector in which the prediction direction is backward;
  • the image block x is subjected to pixel value prediction using a non-translational motion model and the combined motion information unit set i.
  • the determining, by the K1 motion information unit, the combined motion information unit set i includes:
  • each of the N candidate motion information unit sets includes Each of the motion information units is selected from at least a portion of the motion information unit of the candidate motion information unit set corresponding to the different pixel samples of the K1 pixel samples, wherein the N is a positive integer.
  • Each of the N candidate combined motion information unit sets includes K1 motion information units.
  • the set of the N candidate combined motion information units meet the first condition, the second condition, the third condition, At least one of the fourth condition and the fifth condition,
  • the first condition includes any one of the N candidate combined motion information unit sets The motion mode of the image block x indicated by the motion information unit in the candidate merged motion information unit is a non-translation motion
  • the second condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction;
  • the third condition includes that the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same;
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N candidate motion information unit sets is less than or equal to a horizontal component threshold;
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets is less than or equal to a vertical component threshold.
  • the K1 pixel samples include The upper left pixel sample, the upper right pixel sample, the lower left pixel sample, and at least 2 pixel samples of the central pixel sample a1 of the image block x;
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x including the upper left vertex of the image block x;
  • the lower left pixel sample of the image block x a lower left vertex of the image block x or a pixel block in the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex or a location of the image block x a pixel block in the image block x that includes the upper right vertex of the image block x;
  • a central pixel sample a1 of the image block x is a central pixel point of the image block x or the image block x includes the A block of pixels at the center pixel of image block x.
  • the candidate motion information unit set corresponding to the upper left pixel sample of the image block x includes motion information units of x1 pixel samples, wherein the x1 pixel samples include at least one spatial domain with the upper left pixel sample of the image block x Adjacent pixel samples and/or at least one upper left image with the image block x a sample of pixels adjacent to the time domain of the prime sample, wherein x1 is a positive integer;
  • the x1 pixel samples include a pixel sample having the same position as an upper left pixel sample of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the left, a spatially adjacent pixel sample on the upper left of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block x includes motion information units of x2 pixel samples, wherein the x2 pixel samples include at least one airspace phase with the upper right pixel sample of the image block x a neighboring pixel sample and/or at least one pixel sample adjacent to a time domain of an upper right pixel sample of the image block x, the x2 being a positive integer;
  • the x2 pixel samples include a pixel sample having the same position as the upper right pixel sample of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the right side, a spatially adjacent pixel sample on the upper right of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block x includes motion information units of x3 pixel samples, wherein the x3 pixel samples include at least one lower left pixel sample airspace with the image block x An adjacent pixel sample and/or at least one pixel sample adjacent to a time domain of a lower left pixel sample of the image block x, the x3 being a positive integer;
  • the x3 pixel samples include a pixel sample that is the same as a lower left pixel sample position of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the left side, a spatially adjacent pixel sample on the lower left of the image block x, and a spatially adjacent pixel sample on the lower side of the image block x.
  • the set of candidate motion information units corresponding to the central pixel sample a1 of the image block x includes motion information units of x5 pixel samples, wherein one of the x5 pixel samples is a pixel sample a2,
  • the position of the central pixel sample a1 in the video frame to which the image block x belongs is the same as the position of the pixel sample a2 in the adjacent video frame of the video frame to which the image block x belongs, and the x5 is A positive integer.
  • Performing pixel value prediction on the image block x by using the non-translational motion model and the combined motion information unit set i includes: when the prediction direction in the merged motion information unit set i is a motion vector in a first prediction direction If the corresponding reference frame index is different, the merged motion information unit set i is subjected to scaling processing such that the motion vector in the merged motion information unit set i whose prediction direction is the first prediction direction is scaled to the same reference a frame, performing pixel value prediction on the image block x by using a non-translational motion model and a combined motion information unit set i after performing scaling processing, wherein the first prediction direction is forward or backward;
  • the performing pixel value prediction on the image block x by using the non-translational motion model and the combined motion information unit set i includes:
  • the merged motion information unit set i is subjected to scaling processing such that the forward motion vector in the merged motion information unit set i is forwarded to the same reference frame and the merged motion is caused
  • the prediction direction in the information unit set i is that the backward motion vector is scaled to the same reference frame, and the image block x is subjected to pixel value prediction using the non-translating motion model and the merged motion information unit set i subjected to the scaling processing.
  • the ninth possible aspect of the first aspect In combination with the first aspect or the first possible implementation of the first aspect to any one of the possible implementations of the eighth possible implementation of the first aspect, the ninth possible aspect of the first aspect In an embodiment,
  • the method further includes:
  • Determining K2 pixel samples in the image block y determining a set of candidate motion information units corresponding to each of the K2 pixel samples; wherein the K2 is an integer greater than 1, the image block y and The image block x is adjacent to the airspace;
  • the candidate motion information unit set corresponding to each of the K2 pixel samples includes at least one candidate motion information unit, and the candidate motion information unit set corresponding to the pixel sample z1 in the K2 pixel samples.
  • the motion information unit a2 is obtained, and the motion information unit a2 is obtained based on a motion information unit of the pixel sample z2, where the pixel sample z2 is a pixel sample of the image block x whose distance from the pixel sample z1 is less than a threshold value, or
  • the pixel sample z2 is a pixel sample in the image block x that is closest to the pixel sample z1; the K2 motion information units in the merged motion information unit set j are respectively selected from the K2 pixels. At least part of the motion information unit of the candidate motion information unit set corresponding to each pixel sample in the sample that meets the constraint condition;
  • the image block y is subjected to pixel value prediction using a non-translational motion model and the combined motion information unit set j.
  • the non-translational motion model is any one of the following models: an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, a shear motion model, a scale motion model, and a bilinear motion model.
  • the eleventh possible implementation of the first aspect In combination with the first aspect or the first possible implementation of the first aspect to any one of the possible implementations of the tenth possible implementation of the first aspect, the eleventh possible implementation of the first aspect In the way,
  • the performing pixel value prediction on the image block x by using the non-translational motion model and the combined motion information unit set i includes:
  • Calculating the image block x by using a non-translational motion model and the combined motion information unit set i a motion vector of each pixel in the image, using the calculated motion vector of each pixel in the image block x to determine a predicted pixel value of each pixel in the image block x;
  • the twelfth possible aspect of the first aspect In an embodiment,
  • the image prediction method is applied to a video encoding process or the image prediction method is applied to a video decoding process.
  • the N candidate merges Determining the merged motion information unit set i including the K1 motion information units among the motion information unit sets includes: based on the identification of the merged motion information unit set i obtained from the video code stream, from the N candidate combined motion information unit sets The merged motion information unit set i including K1 motion information units is determined.
  • Determining the merged motion information unit set i including the K1 motion information units among the motion information unit sets includes: determining, from the N candidate motion information unit sets, the K1 motion information units according to the distortion or rate distortion cost Combine the motion information unit set i.
  • the image prediction method is applied to In the case of a video encoding process, the method further comprises writing the identification of the combined motion information element set i to the video code stream.
  • a second aspect of the present invention provides an image prediction apparatus, including:
  • a first determining unit configured to determine K1 pixel samples in the image block x, and determine the K1 pixels a set of candidate motion information units corresponding to each pixel sample in the sample; the candidate motion information unit set corresponding to each pixel sample includes at least one motion information unit of the candidate, and the K1 is an integer greater than or equal to 2. ;
  • a second determining unit configured to determine a merged motion information unit set i including K1 motion information units, wherein each of the motion information unit sets i is selected from the K1 pixel samples respectively At least part of the motion information unit in the candidate motion information unit set corresponding to the different pixel samples, wherein the motion information unit includes a motion vector in which the prediction direction is a forward direction and/or a motion direction in which the prediction direction is a backward direction;
  • a prediction unit configured to perform pixel value prediction on the image block x by using the non-translational motion model and the combined motion information unit set i.
  • the second determining unit is specifically configured to: determine, from among the N candidate motion information unit sets, a combined motion information unit set i that includes K1 motion information units; where the N candidate combined motion information unit sets Each of the motion information units included in each candidate merged motion information unit set is selected from at least a part of the motion information unit of the candidate motion information unit set corresponding to the different pixel samples in the K1 pixel samples, respectively.
  • the N is a positive integer, and the set of N candidate combined motion information units are different from each other, and each set of candidate combined motion information units in the N candidate combined motion information unit sets includes K1 motion information units.
  • the set of the N candidate combined motion information units meet the first condition, the second condition, the third condition, At least one of the fourth condition and the fifth condition,
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion
  • the second condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction;
  • the third condition includes that the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same;
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N candidate motion information unit sets is less than or equal to a horizontal component threshold;
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets is less than or equal to a vertical component threshold.
  • the K1 pixel samples include The upper left pixel sample, the upper right pixel sample, the lower left pixel sample, and at least 2 pixel samples of the central pixel sample a1 of the image block x;
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x including the upper left vertex of the image block x;
  • the lower left pixel sample of the image block x a lower left vertex of the image block x or a pixel block in the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex or a location of the image block x a pixel block in the image block x that includes the upper right vertex of the image block x;
  • a central pixel sample a1 of the image block x is a central pixel point of the image block x or the image block x includes the A block of pixels at the center pixel of image block x.
  • the candidate motion information unit set corresponding to the upper left pixel sample of the image block x includes motion information units of x1 pixel samples, wherein the x1 pixel samples include at least one spatial domain with the upper left pixel sample of the image block x a neighboring pixel sample and/or at least one pixel sample adjacent to a time domain of an upper left pixel sample of the image block x, the x1 being a positive integer;
  • the x1 pixel samples include a pixel sample having the same position as an upper left pixel sample of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the left, a spatially adjacent pixel sample on the upper left of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block x includes motion information units of x2 pixel samples, wherein the x2 pixel samples include at least one airspace phase with the upper right pixel sample of the image block x a neighboring pixel sample and/or at least one pixel sample adjacent to a time domain of an upper right pixel sample of the image block x, the x2 being a positive integer;
  • the x2 pixel samples include a pixel sample having the same position as the upper right pixel sample of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the right side, a spatially adjacent pixel sample on the upper right of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block x includes motion information units of x3 pixel samples, wherein the x3 pixel samples include at least one lower left pixel sample airspace with the image block x An adjacent pixel sample and/or at least one pixel sample adjacent to a time domain of a lower left pixel sample of the image block x, the x3 being a positive integer;
  • the x3 pixel samples include a pixel sample that is the same as a lower left pixel sample position of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the left side, a spatially adjacent pixel sample on the lower left of the image block x, and a spatially adjacent pixel sample on the lower side of the image block x.
  • the set of candidate motion information units corresponding to the central pixel sample a1 of the image block x includes motion information units of x5 pixel samples, wherein one of the x5 pixel samples is a pixel sample a2,
  • the position of the central pixel sample a1 in the video frame to which the image block x belongs is the same as the position of the pixel sample a2 in the adjacent video frame of the video frame to which the image block x belongs, and the x5 is A positive integer.
  • the predicting unit is specifically configured to: when a prediction direction in the merged motion information unit set i is a first prediction If the reference frame index corresponding to the motion vector of the direction is different, the merged motion information unit set i is subjected to a scaling process such that the motion vector in the merged motion information unit set i is the motion direction of the first prediction direction is Zooming to the same reference frame, performing pixel value prediction on the image block x by using a non-translating motion model and a combined motion information unit set i subjected to scaling processing, wherein the first prediction direction is forward or backward;
  • the prediction unit is specifically configured to: when the prediction direction in the merged motion information unit set i is a reference frame index corresponding to a forward motion vector, and the prediction direction in the merged motion information unit set i is backward If the reference frame index corresponding to the motion vector is different, the merged motion information unit set i is subjected to scaling processing such that the forward motion vector in the merged motion information unit set i is scaled to the same motion vector Referencing the frame and causing the backward direction motion vector in the merged motion information unit set i to be scaled to the same reference frame, using the non-translational motion model and the scaled motion information unit set i Image block x performs pixel value prediction.
  • the prediction unit is specifically configured to calculate a motion vector of each pixel in the image block x by using a non-translating motion model and the combined motion information unit set i, and using the calculated image block x
  • the motion vector of each pixel in the image determines the predicted pixel value of each pixel in the image block x
  • the prediction unit is specifically configured to calculate a motion vector of each pixel block in the image block x by using a non-translating motion model and the combined motion information unit set i, by using the calculated image block x
  • the motion vector of each pixel block determines a predicted pixel value of each pixel of each pixel block in the image block x.
  • the first determining unit is further configured to determine K2 pixel samples in the image block y, and determine a candidate motion information unit set corresponding to each of the K2 pixel samples; wherein the K2 is greater than An integer of 1, the image block y is adjacent to the image block x spatial domain, and the candidate motion information unit set corresponding to each of the K2 pixel samples includes at least one motion information unit of the candidate;
  • the second determining unit is further configured to: determine a combined motion information unit set j that includes K2 motion information units, where the candidate motion information unit set corresponding to the pixel samples z1 in the K2 pixel samples includes motion An information unit a2, the motion information unit a2 is obtained based on a motion information unit of the pixel sample z2, wherein the pixel sample z2 is a pixel sample in the image block x that is less than a threshold value from the pixel sample z1, or a pixel sample z2 is a pixel sample in the image block x that is closest to the pixel sample z1; the K2 motion information units in the merged motion information unit set j are respectively selected from the K2 pixel samples. At least part of the motion information unit of the candidate motion information unit set corresponding to each pixel sample that meets the constraint condition;
  • the prediction unit is further configured to perform pixel value prediction on the image block y by using a non-translating motion model and the combined motion information unit set j.
  • the non-translational motion model is any one of the following models: an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, a shear motion model, a scale motion model, and a bilinear motion model.
  • the image prediction device is applied to a video encoding device or the image prediction device is applied to a video decoding device.
  • the second determining unit is specific And determining, for the distortion or rate distortion cost, the merged motion information unit set i including the K1 motion information units from among the N candidate combined motion information unit sets.
  • the prediction unit when the image prediction apparatus is applied, is further configured to write the identifier of the combined motion information unit set i into the video code stream.
  • the second determining The unit is specifically configured to determine, according to the identifier of the merged motion information unit set i obtained from the video code stream, a merged motion information unit set i including K1 motion information units from among the N candidate combined motion information unit sets.
  • pixel value prediction is performed on the image block x by using the non-translating motion model and the combined motion information unit set i, and each of the motion information unit sets i is merged.
  • the motion information units are respectively selected from at least part of the motion information units in the candidate motion information unit set corresponding to the different pixel samples in the K1 pixel samples, and the traditional motion is discarded because the selection range of the combined motion information unit set i is relatively small.
  • a mechanism for screening out a motion information unit of K1 pixel samples by a large number of calculations in all possible candidate motion information element sets of K1 pixel samples which is advantageous for improving coding efficiency and also for reducing non-based non-
  • the translational motion model performs the computational complexity of image prediction, which makes it possible to introduce a non-translational motion model into the video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • FIG. 1 is a schematic diagram of a prediction unit division manner corresponding to intra prediction according to an embodiment of the present invention
  • FIG. 1 is a schematic diagram of several prediction unit division manners corresponding to inter prediction according to an embodiment of the present disclosure
  • FIG. 1 is a schematic flowchart of an image prediction method according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of another image prediction method according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of a plurality of candidate motion information unit sets for determining pixel samples according to an embodiment of the present disclosure
  • 2 e is a schematic diagram of vertex coordinates of an image block x according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of the affine motion of a pixel according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of bilinear interpolation provided by an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of motion vector storage according to an embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of another video encoding method according to an embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of another video decoding method according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of an image prediction apparatus according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of another image prediction apparatus according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of another image prediction apparatus according to an embodiment of the present invention.
  • Embodiments of the present invention provide an image prediction method and related apparatus, in order to reduce computational complexity of image prediction based on a non-translational motion model.
  • the video sequence consists of a series of pictures (English: picture), the picture is further divided into slices (English: slice), and the slice is further divided into blocks (English: block).
  • the video coding is performed in units of blocks, and can be encoded from left to right and from top to bottom line from the upper left corner position of the picture.
  • the concept of block is further extended.
  • the MB can be further divided into a plurality of prediction blocks (English: partition) that can be used for predictive coding.
  • coding unit English: coding unit, abbreviation: CU
  • prediction unit English: prediction unit, abbreviation: PU
  • transform unit English: transform unit, abbreviation: TU
  • CU PU or TU
  • the PU can correspond to a prediction block and is a basic unit of predictive coding.
  • the CU is further divided into a plurality of PUs according to a division mode.
  • the TU can correspond to a transform block and is a basic unit for transforming the prediction residual.
  • High-performance video coding English: high efficiency video coding, abbreviated: HEVC
  • coding tree block English: coding tree block, abbreviated: CTB
  • the size of a coding unit may include four levels of 64 ⁇ 64, 32 ⁇ 32, 16 ⁇ 16, and 8 ⁇ 8, and coding units of each level may be divided into different according to intra prediction and inter prediction.
  • the size of the prediction unit. 1 - a and FIG. Corresponding prediction unit The way of division.
  • the skip mode and the direct mode become effective tools for improving coding efficiency, and are used at low bit rates.
  • the blocks of these two coding modes can account for more than half of the entire coding sequence.
  • the skip mode only one skip mode flag needs to be passed in the code stream, and the motion vector of the current image block can be derived by using the peripheral motion vector, and the value of the reference block is directly copied according to the motion vector as the current image block. Reconstruction value.
  • the encoder can derive the motion vector of the current image block by using the peripheral motion vector, and directly copy the value of the reference block as the predicted value of the current image block according to the motion vector, and use the predicted value pair at the encoding end.
  • the current image block is encoded and predicted.
  • HEVC high-performance video coding
  • AMVP adaptive motion vector prediction
  • the fusion coding constructs a candidate motion information set by using motion information of the coded block around the current coding block (including motion vector (audio vector) and prediction direction and reference frame index, etc.), by comparing
  • the candidate motion information with the highest coding efficiency may be selected as the motion information of the current coding block, the prediction value of the current coding block is found in the reference frame, the current coding block is predictively coded, and at the same time, the peripheral coded block from which the representation is selected is selected.
  • the index value of the motion information is written to the code stream.
  • the adaptive motion vector prediction mode when used, by using the motion vector of the peripheral coded block as the predicted value of the current coded block motion vector, a motion vector with the highest coding efficiency may be selected to predict the motion vector of the current coded block, and An index value indicating which peripheral motion vector is selected is written to the video stream.
  • the image prediction method provided by the embodiment of the present invention is described below.
  • the execution body of the image prediction method provided by the embodiment of the present invention is a video coding device or a video decoding device, where the video coding device or the video decoding device may be any output or A device that stores video, such as a laptop, tablet, PC, cell phone, or video server.
  • the image prediction method includes: determining K1 pixel samples in the image block x, and determining a candidate motion information unit set corresponding to each of the K1 pixel samples And wherein the K1 is an integer greater than or equal to 2, the candidate motion information unit set corresponding to each pixel sample includes at least one motion information unit of the candidate; and the combined motion information unit set including the K1 motion information unit is determined.
  • Each of the motion information unit sets i is selected from at least a portion of motion information units in a candidate motion information unit set corresponding to different ones of the K1 pixel samples, respectively.
  • the motion information unit includes a motion vector in which the prediction direction is a forward motion vector and/or a prediction direction is a backward motion vector; the image block x is subjected to pixel value prediction using the non-transition motion motion model and the combined motion information unit set i.
  • FIG. 1-c is a schematic flowchart of an image prediction method according to an embodiment of the present invention.
  • an image prediction method provided by an embodiment of the present invention may include:
  • the candidate motion information unit set corresponding to each pixel sample includes at least one motion information unit of the candidate.
  • K1 is an integer greater than one.
  • the K can be equal to 2, 3, 4 or other values.
  • the manner in which the K1 pixel samples are selected may be various.
  • the pixel samples mentioned in the embodiments of the present invention may be pixel points or pixel blocks including at least two pixel points.
  • referring to the motion information unit in each embodiment of the present invention may include a motion vector in which the prediction direction is forward and/or a motion vector in which the prediction direction is backward. That is, one motion information unit may include one motion vector or two motion vectors that may include different prediction directions.
  • the prediction direction corresponding to the motion information unit is forward, it indicates that the motion information unit includes a motion vector whose prediction direction is forward, but does not include a motion vector whose prediction direction is backward. If the prediction direction corresponding to the motion information unit is backward, it indicates that the motion information unit includes a motion vector whose prediction direction is backward but does not include a motion vector whose prediction direction is forward. If the prediction direction corresponding to the motion information unit is unidirectional, it indicates that the motion information unit includes a forward motion vector but does not include a prediction direction. It is a backward motion vector, or indicates that the motion information unit includes a motion vector in which the prediction direction is backward but does not include a motion vector in which the prediction direction is forward. Wherein, if the prediction direction corresponding to the motion information unit is bidirectional, the motion information unit includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward.
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and at least 2 pixel samples of the central pixel sample a1 of the image block x.
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x containing the upper left vertex of the image block x; the lower left pixel of the image block x
  • the sample is a lower left vertex of the image block x or a pixel block of the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex of the image block x or a pixel block in the image block x that includes an upper right vertex of the image block x;
  • a central pixel sample a1 of the image block x is a central pixel point of the image block x or an inclusion in the image block x A block of pixels of a central pixel of image block x.
  • the size of the pixel block is, for example, 2*2, 1*2, 4*2, 4*4, or other size.
  • Each of the motion information units i is selected from at least a part of motion information units in a candidate motion information unit set corresponding to different ones of the K1 pixel samples. That is, any two of the motion information unit sets i are selected from at least a part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples of the K1 pixel samples.
  • K1 is equal to 3, wherein K1 pixel samples include pixel sample 001, pixel sample 002, and pixel sample 003.
  • the candidate motion information unit set corresponding to the pixel sample 001 is the candidate motion information unit set 011.
  • the candidate motion information unit set corresponding to the pixel sample 002 is the candidate motion information unit set 022.
  • the candidate motion information unit set corresponding to the pixel sample 003 is the candidate motion information unit set 033.
  • the merged motion information unit set i includes a motion information unit C01, a motion information unit C02, and a motion information unit C03, wherein the motion information unit C01 may be selected from the candidate motion information unit set 011, and the motion information unit C02 may be selected from the candidate motion information.
  • the candidate motion information unit set 033 can be selected. That is to say, any two motion information units of the motion information unit C01, the motion information unit C02, and the motion information unit C03 are selected from the candidate motion information unit set 011, the candidate motion information unit set 022, and the candidate motion information unit. At least a portion of the motion information elements in the set of different candidate motion information elements in set 033.
  • K1 is equal to other values and so on.
  • the merged motion information unit set i includes the motion information unit C01, the motion information unit C02, and the motion information unit C03, and any one of the motion information unit C01, the motion information unit C02, and the motion information unit C03 may include a prediction.
  • the direction of the forward motion vector and/or the prediction direction is a backward motion vector. Therefore, the combined motion information unit set i may include three motion vectors (the prediction manners corresponding to the three motion vectors may be forward or backward) Or the three motion vectors may include two motion vectors whose prediction direction is forward and one motion vector whose prediction direction is backward, or may also include two motion vectors whose prediction direction is forward and the prediction direction is backward.
  • the method includes a motion vector with a forward direction of prediction and three motion vectors with a backward direction of prediction, or may also include three motion vectors and a forward direction.
  • the motion direction is a backward motion vector
  • the motion direction may also include five motion vectors (the five motion vectors may include two motion vectors whose prediction direction is forward and three motion vectors whose prediction direction is backward, or It may also include three motion vectors whose prediction direction is forward and two motion vectors whose prediction direction is backward, and may also include six motion vectors (the six motion vectors include three motion vectors whose prediction direction is forward) And the prediction direction is backward 3 motion vectors).
  • the image block x can be regarded as a current image block, and the current image block can be a current coding block or a current decoding block.
  • the non-translational motion model is any one of the following models: an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, and a shear motion. Models, scaled motion models, and bilinear motion models. It can be understood that the non-translational motion model is not limited to the specific examples described above.
  • the pixel value prediction is performed on the image block x by using the non-translating motion model and the combined motion information unit set i, and each motion information in the motion information unit set i is combined.
  • the unit is respectively selected from at least part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples, and the selected range of the combined motion information unit set i is relatively small, and the traditional technology is adopted.
  • the mechanism of filtering out one motion information unit of K1 pixel samples by a large number of calculations in all possible candidate motion information element sets of K1 pixel samples is beneficial to improve coding efficiency and also to reduce non-translational motion based on
  • the model performs the computational complexity of image prediction, which in turn makes it possible to introduce a non-translational motion model into the video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • the image prediction method provided by this embodiment may be applied to a video encoding process or may be applied to a video decoding process.
  • the manner of determining the combined motion information unit set i including K1 motion information units may be varied.
  • determining the merged motion information unit set i that includes the K1 motion information units may include: determining, by the N candidate motion information unit sets, K1 motions a merged motion information unit set i of the information unit; wherein each motion information unit included in each of the N candidate motion information unit sets is selected from the K1 pixel samples At least part of the motion information unit that meets the constraint in the candidate motion information unit set corresponding to the different pixel samples, the N is a positive integer, the N candidate combined motion information unit sets are different from each other, and the N candidate merges Each candidate merged motion information unit set in the motion information unit set includes K1 motion information units.
  • the two candidate combined motion information unit sets are different, and may be that the motion information units included in the candidate combined motion information unit set are not completely the same.
  • the two motion information units are different, and may refer to different motion vectors included in the two motion information units, or different motion directions corresponding to the motion vectors included in the two motion information units, or included in the two motion information units.
  • the motion vector corresponds to a different reference frame index.
  • the two motion information units are the same, which may mean that the motion vectors included in the two motion information units are the same, and the two motion information
  • the motion vectors included in the unit correspond to the same prediction direction, and the reference frames corresponding to the motion vectors included in the two motion information units are the same.
  • the merged motion information unit set i may include: determining a combined motion information unit including K1 motion information units from among the N candidate combined motion information unit sets based on the identification of the merged motion information unit set i obtained from the video code stream Set i.
  • the method may further include: writing the identifier of the merged motion information unit set i Into the video stream.
  • determining the combined motion information unit set i that includes K1 motion information units from among the N candidate combined motion information unit sets may include: based on distortion or rate distortion cost A merged motion information unit set i including K1 motion vectors is determined from among the N candidate combined motion information unit sets.
  • the rate distortion cost corresponding to the merged motion information unit set i is less than or equal to the rate distortion cost corresponding to any one of the combined motion information unit sets except the combined motion information unit set i in the N candidate motion information unit sets. .
  • the distortion corresponding to the merged motion information unit set i is less than or equal to the distortion corresponding to any one of the combined motion information unit sets except the combined motion information unit set i in the N candidate motion information unit sets.
  • the rate-distortion cost corresponding to a certain candidate combined motion information unit set may be, for example, utilized.
  • the distortion corresponding to a certain candidate combined motion information unit set (for example, the combined motion information unit set i in the N candidate motion information unit sets) in the foregoing N candidate combined motion information unit sets, for example, may be an image block.
  • the original pixel value (such as image block x) is merged with the candidate a set of motion information units (eg, merged motion information unit set i) between the predicted pixel values of the image block obtained by performing pixel value prediction on the image block (ie, between the original pixel value of the image block and the predicted pixel value) Distortion).
  • the original pixel value of the image block (such as image block x) and the pixel value prediction of the image block by using the certain candidate combined motion information unit set (for example, the combined motion information unit set i)
  • the resulting distortion between the predicted pixel values of the image block may be, for example, an original pixel value of the image block (eg, image block x) and a set of combined motion information units using the certain candidate (eg, a combined motion information unit set) i) a sum of square differences or (SAD, sum of absolution differences) or an error and/or between the predicted pixel values of the image block obtained by performing pixel value prediction on the image block
  • Other distortion parameters that measure distortion.
  • N is a positive integer.
  • N described above may be, for example, equal to 1, 2, 3, 4, 5, 6, 8, or other values.
  • each motion information unit in any one of the N candidate motion information unit sets may be different from each other.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition. condition.
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the motion mode of the image block x indicated by the motion information unit in the candidate combined motion information unit set may be considered as a translational motion, and vice versa, the motion of the image block x indicated by the motion information unit in the candidate combined motion information unit is considered to be a non-translational motion, wherein the first prediction direction is forward or backward .
  • the candidate may be considered as the candidate.
  • the motion mode of the image block x indicated by the motion information unit in the merged motion information unit is a translational motion, and vice versa, the image block indicated by the motion information unit in the candidate combined motion information unit may be considered.
  • the mode of motion of x is non-translational motion.
  • the second condition may include that the prediction directions corresponding to the at least two motion information units in any one of the N candidate motion information unit sets are the same. For example, when both motion information units include a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward, it indicates that the prediction directions corresponding to the two motion information units are the same. For another example, when one of the two motion information units includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward, the other motion information unit includes a motion vector in which the prediction direction is forward.
  • the motion vector with the prediction direction being backward is not included, or the motion information unit includes the motion vector with the prediction direction being the backward direction but not the motion vector with the prediction direction being the forward direction, which may represent the prediction corresponding to the two motion information units.
  • the direction is different.
  • one of the two motion information units includes a motion vector whose prediction direction is a forward motion but does not include a motion vector whose backward direction is a backward direction
  • another motion information unit includes a backward direction of the prediction direction.
  • the motion vector, but not including the forward direction motion vector may indicate that the prediction directions corresponding to the two motion information units are different.
  • both motion information units include a motion vector whose prediction direction is forward, but the two motion information units do not include the motion vector whose prediction direction is backward, indicating that the prediction directions of the two motion information units are the same.
  • both motion information units include a motion vector whose prediction direction is backward, but the two motion information units do not include a motion vector whose prediction direction is forward, indicating that the prediction directions of the two motion information units are the same.
  • the third condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same reference frame index.
  • both motion information units include a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward
  • the prediction direction in the two motion information units is a reference corresponding to the forward motion vector.
  • the frame index is the same
  • the prediction direction in the two motion information units is the same as the reference frame index corresponding to the backward motion vector, which may indicate that the reference frame indexes corresponding to the two motion information units are the same.
  • the other motion information unit when one of the two motion information units includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward, the other motion information unit includes a prediction direction. a forward motion vector but does not include a motion vector whose prediction direction is backward, or the other motion information unit includes a motion vector in which the prediction direction is backward but does not include a motion vector in which the prediction direction is forward, indicating the two The prediction direction corresponding to the motion information unit is different, and may indicate that the reference frame indexes corresponding to the two motion information units are different.
  • the other motion information unit when one of the two motion information units includes a motion vector in which the prediction direction is a forward motion but does not include a motion vector in which the prediction direction is backward, the other motion information unit includes a motion in which the prediction direction is backward.
  • the vector, but not including the forward direction motion vector may indicate that the reference frame indices corresponding to the two motion information units are different.
  • the other motion information unit when one of the two motion information units includes a motion vector in which the prediction direction is a forward motion but does not include a motion vector in which the prediction direction is backward, the other motion information unit includes a motion in which the prediction direction is forward.
  • the vector does not include the motion vector whose prediction direction is backward, and the prediction direction in the two motion information units is the same as the reference frame index corresponding to the forward motion vector, and may represent the reference frame corresponding to the two motion information units.
  • the index is different.
  • the other motion information unit includes a motion in which the prediction direction is backward.
  • the vector does not include the motion vector of the forward direction, and the reference frame in the two motion information units is the same as the reference frame index corresponding to the backward motion vector, and may represent the reference frame corresponding to the two motion information units.
  • the index is different.
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a horizontal component threshold.
  • the horizontal component threshold example may be equal to 1/3 of the width of the image block x, 1/2 of the width of the image block x, 2/3 of the width of the image block x, or 3/4 of the width of the image block x or the like. size.
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets is less than or equal to a vertical component threshold.
  • a vertical component threshold may be equal to 1/3 of the height of the image block x, 1/2 of the height of the image block x, 2/3 of the height of the image block x, or 3/4 of the height of the image block x or the like. size.
  • the upper left pixel sample of the image block x The corresponding set of candidate motion information units includes motion information units of x1 pixel samples, wherein the x1 pixel samples include at least one pixel sample adjacent to an upper left pixel sample spatial domain of the image block x and/or at least one The pixel sample adjacent to the time domain of the upper left pixel sample of the image block x, the x1 being a positive integer.
  • the x1 pixel samples include only at least one pixel sample adjacent to the upper left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper left pixel sample time domain of the image block x.
  • x1 above may be equal to 1, 2, 3, 4, 5, 6, or other values, for example.
  • the x1 pixel samples include a pixel sample having the same position as an upper left pixel sample of the image block x among video frames adjacent to a video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the left, a spatially adjacent pixel sample on the upper left of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block x includes motion information units of x2 pixel samples, where the x2 pixel samples include At least one pixel sample adjacent to the upper right pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper right pixel sample time domain of the image block x, the x2 being a positive integer.
  • x2 above may be, for example, equal to 1, 2, 3, 4, 5, 6, or other values.
  • the x2 pixel samples include a pixel sample having the same position as the upper right pixel sample of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the right side, a spatially adjacent pixel sample on the upper right of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block x includes motion information units of x3 pixel samples, where the x3 pixel samples include At least one pixel sample adjacent to a lower left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to a lower left pixel sample time domain of the image block x, the x3 being a positive integer.
  • the x3 pixel samples include only at least one pixel sample adjacent to the lower left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the lower left pixel sample time domain of the image block x.
  • x3 above may be equal to 1, 2, 3, 4, 5, 6, or other values, for example.
  • the x3 pixel samples include a pixel sample identical to a position of a lower left pixel sample of the image block x among video frames adjacent to a time domain of a video frame to which the image block x belongs, the image block x At least one of a spatially adjacent pixel sample on the left side, a spatially adjacent pixel sample on the lower left of the image block x, and a spatially adjacent pixel sample on the lower side of the image block x.
  • the set of candidate motion information units corresponding to the central pixel sample a1 of the image block x includes motion information units of x5 pixel samples, where the x5 pixel samples are included.
  • One of the pixel samples is the pixel sample a2.
  • the x5 pixel samples include only pixel samples a2.
  • the position of the central pixel sample a1 in the video frame to which the image block x belongs is the same as the position of the pixel sample a2 in the adjacent video frame of the video frame to which the image block x belongs, and the x5 is A positive integer.
  • performing pixel value prediction on the image block x by using the non-translational motion model and the combined motion information unit set i may include: when the merge motion In a case where the prediction direction in the information unit set i is different from the reference frame index corresponding to the motion vector in the first prediction direction, the merged motion information unit set i is subjected to scaling processing so that the merged motion information unit set i is The motion vector whose prediction direction is the first prediction direction is scaled to the same reference frame, and the pixel value prediction is performed on the image block x by using the non-translating motion model and the merged motion information unit set i after performing the scaling process.
  • the first prediction direction is forward or backward.
  • performing pixel value prediction on the image block x by using the non-translating motion model and the combined motion information unit set i may include: when the prediction direction in the combined motion information unit set i is a forward motion vector corresponding If the reference frame index is different, and the prediction direction in the merged motion information unit set i is different from the reference frame index corresponding to the backward motion vector, the merged motion information unit set i is subjected to scaling processing so that The prediction direction in the merged motion information unit set i is that the forward motion vector is scaled to the same reference frame and the backward motion vector in the merged motion information unit set i is scaled to the same reference frame And performing pixel value prediction on the image block x by using the non-translational motion model and the merged motion information unit set i after performing the scaling process.
  • each image block in the current video frame it can be followed by the image block x
  • the corresponding pixel value prediction method performs pixel value prediction in a similar manner.
  • some image blocks in the current video frame may also perform pixel value prediction in a manner different from the pixel value prediction manner corresponding to the image block x.
  • performing pixel value prediction on the image block x by using the non-motion motion model and the combined motion information unit set i may include: performing the merge motion The motion vector in the information unit set i performs motion estimation processing to obtain the combined motion information unit set i after the motion estimation processing, and the combined motion information unit set i using the non-translational motion model and the motion estimation processing x performs pixel value prediction.
  • performing pixel value prediction on the image block x by using the non-translational motion model and the merged motion information unit set i after performing the scaling process may include: Performing motion estimation processing on the motion vector in the merged motion information unit set i after the scaling process to obtain the combined motion information unit set i after the motion estimation processing, and the combined motion information unit after the motion estimation processing and the motion estimation processing The set i performs pixel value prediction on the image block x.
  • the performing pixel value prediction on the image block x by using the non-translational motion model and the combined motion information unit set i includes: utilizing non-translation The motion model and the combined motion information unit set i calculate a motion vector of each pixel in the image block x, and determine the image block x by using the calculated motion vector of each pixel in the image block x. a predicted pixel value for each pixel in ;
  • the motion vector of each pixel block in the image block x is calculated by using the non-translational motion model and the combined motion information unit set i, and the calculated motion vector of each pixel block in the image block x is calculated.
  • a predicted pixel value of each pixel of each pixel block in the image block x is determined.
  • the test finds that if the motion vector of each pixel block in the image block x is first calculated by using the non-translational motion model and the combined motion information unit set i, then each of the calculated image blocks x is used.
  • the motion vector of the pixel block determines the predicted pixel value of each pixel point of each pixel block in the image block x, since the calculation of the motion vector is performed by using the pixel block in the image block x as a granularity, which is advantageous for a large reduction Computational complexity.
  • the method further includes: determining K2 pixel samples in the image block y, where K2 is an integer greater than 1 (K1 may be equal to or not equal to K2); a set of candidate motion information units corresponding to each of the K2 pixel samples; wherein the K2 is an integer greater than 1, the image block y is adjacent to the image block x spatial domain; and the K2 motions are determined to be included a merged motion information unit set j of the information unit; wherein the candidate motion information unit set corresponding to each of the K2 pixel samples includes at least one motion information unit of the candidate, and the pixels in the K2 pixel samples
  • the candidate motion information unit set corresponding to the sample z1 includes a motion information unit a2, which is obtained based on a motion information unit of the pixel sample z2, the pixel sample z2 being the image block x and the pixel sample z1 a pixel sample having a distance smaller than a threshold, or the pixel sample z2
  • the K2 pixel samples may further include at least two of an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and a central pixel sample a3 of the image block y.
  • an upper left pixel sample of the image block y is an upper left vertex of the image block y or a pixel block in the image block y including an upper left vertex of the image block y;
  • the image block y The lower left pixel sample is the lower left vertex of the image block y or the pixel block of the image block y including the lower left vertex of the image block y;
  • the upper right pixel sample of the image block y is the upper right of the image block y a vertex or a pixel block in the image block y containing the upper right vertex of the image block y;
  • a central pixel sample a3 of the image block y is a central pixel point of the image block y or in the image block y A block of pixels containing a central pixel of the image block y.
  • the manner of determining the combined motion information unit set j including K2 motion information units may be varied.
  • determining the combined motion information unit set j that includes the K2 motion information units may include: determining, by the N2 candidate combined motion information unit sets, the K2 motions a combined motion information unit set j of the information unit; wherein each motion information unit included in each of the N2 candidate combined motion information unit sets is selected from the K2 pixel samples Candidate motion letter corresponding to different pixel samples At least part of the motion information unit that meets the constraint in the information unit set, the N2 candidate combined motion information unit sets are different from each other, and each candidate merged motion information unit set in the N2 candidate combined motion information unit set includes K2 The motion information unit, the N2 being a positive integer.
  • the merged motion information unit set j may include: determining merge motion information including K2 motion information units from among the N2 candidate combined motion information unit sets based on the identifier of the merged motion information unit set j obtained from the video code stream Unit set j.
  • the method when the image prediction method is applied to a video encoding process, the method further includes: writing an identifier of the merged motion information unit set j Video stream.
  • determining the combined motion information unit set j that includes K2 motion information units from among the N2 candidate combined motion information unit sets may include: based on distortion or rate distortion cost Among the N2 candidate combined motion information unit sets, a combined motion information unit set j including K2 motion vectors is determined.
  • the rate distortion cost corresponding to the combined motion information unit set j is less than or equal to the rate distortion cost corresponding to any one of the combined motion information unit sets except the combined motion information unit set j in the N2 candidate combined motion information unit sets.
  • the distortion corresponding to the combined motion information unit set j is less than or equal to the distortion corresponding to any one of the combined motion information unit sets except the combined motion information unit set j in the N2 candidate combined motion information unit sets.
  • the rate-distortion cost corresponding to a certain candidate combined motion information unit set may be utilized, for example, in the foregoing N2 candidate combined motion information unit sets.
  • the set of candidate motion information units (such as the combined motion information unit set j in the N2 candidate combined motion information unit sets) of the N2 candidate combined motion information unit sets may be the original pixel value of the image block (such as image block y) and the pixel value prediction of the image block using the certain candidate combined motion information unit set (such as the merged motion information unit set j). Distortion between predicted pixel values of the image block (ie, distortion between the original pixel value of the image block and the predicted pixel value).
  • the original pixel value of the image block (such as image block y) and the pixel value prediction of the image block by using the certain candidate combined motion information unit set (for example, the combined motion information unit set j)
  • the resulting distortion between the predicted pixel values of the image block may be, for example, an original pixel value of the image block (eg, image block y) and a set of combined motion information units using the certain candidate (eg, a combined motion information unit set) j) a squared error sum (SSD) or absolute error sum (SAD) or error between the predicted pixel values of the image block obtained by performing pixel value prediction on the image block and or other distortion parameters capable of measuring distortion.
  • SSD squared error sum
  • SAD absolute error sum
  • N2 is a positive integer.
  • N2 above may be equal to 1, 2, 3, 4, 5, 6, 8, or other values, for example.
  • the set of N2 candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition. condition.
  • the first condition includes that the motion mode of the image block y indicated by the motion information unit in any one of the N2 candidate combined motion information unit sets is a non-translational motion.
  • the second condition may include that the prediction directions corresponding to the at least two motion information units in any one of the N2 candidate combined motion information unit sets are the same.
  • the third condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N2 candidate combined motion information unit sets have the same reference frame index.
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N2 candidate combined motion information unit sets is less than or equal to a horizontal component threshold.
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N2 candidate combined motion information unit sets is less than or equal to a vertical component threshold.
  • the candidate motion information unit set corresponding to the upper left pixel sample of the image block y includes motion information units of the y1 pixel samples, where
  • the y1 pixel samples include at least one pixel sample adjacent to the upper left pixel sample spatial domain of the image block y and/or at least one pixel sample adjacent to the upper left pixel sample time domain of the image block y, the y1 being A positive integer.
  • the y1 pixel samples include only at least one pixel sample adjacent to the upper left pixel sample spatial domain of the image block y and/or at least one pixel sample adjacent to the upper left pixel sample time domain of the image block y.
  • y1 above may be equal to 1, 2, 3, 4, 5, 6, or other values, for example.
  • the y1 pixel samples include a pixel sample having the same position as an upper left pixel sample of the image block y among video frames adjacent to a video frame time domain to which the image block y belongs, the image block y At least one of a spatially adjacent pixel sample on the left, a spatially adjacent pixel sample on the upper left of the image block y, and a spatially adjacent pixel sample on the upper side of the image block y.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block y includes motion information units of y2 pixel samples, where the y2 pixel samples include At least one pixel sample adjacent to the upper right pixel sample spatial domain of the image block y and/or at least one pixel sample adjacent to the upper right pixel sample time domain of the image block y, the y2 being a positive integer.
  • y2 above may be equal to 1, 2, 3, 4, 5, 6, or other values, for example.
  • the y2 pixel samples include a pixel sample having the same position as the upper right pixel sample of the image block y among the video frames adjacent to the video frame time domain to which the image block y belongs, the image block y At least one of a spatially adjacent pixel sample on the right side, a spatially adjacent pixel sample on the upper right of the image block y, and a spatially adjacent pixel sample on the upper side of the image block y.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block y includes motion information units of y3 pixel samples, where the y3 pixel samples include At least one pixel sample adjacent to a lower left pixel sample spatial domain of the image block y and/or at least one pixel sample adjacent to a lower left pixel sample time domain of the image block y, the y3 being a positive integer.
  • the y3 pixel samples include only at least one pixel sample adjacent to the lower left pixel sample spatial domain of the image block y and/or at least one pixel sample adjacent to the lower left pixel sample time domain of the image block y.
  • y3 above may be equal to 1, 2, 3, 4, 5, 6, or other values, for example.
  • the y3 pixel samples include a pixel sample having the same position as a lower left pixel sample of the image block y among video frames adjacent to a video frame time domain to which the image block y belongs, the image block y At least one of a spatially adjacent pixel sample on the left, a spatially adjacent pixel sample on the lower left of the image block y, and a spatially adjacent pixel sample on the lower side of the image block y.
  • the set of candidate motion information units corresponding to the central pixel sample a3 of the image block y includes motion information units of y5 pixel samples, where the y5 pixel samples are included.
  • One of the pixel samples is the pixel sample a4.
  • the y5 pixel samples include only pixel samples a4.
  • the position of the central pixel sample a3 in the video frame to which the image block y belongs is the same as the position of the pixel sample a4 in the adjacent video frame of the video frame to which the image block y belongs, and the y5 is A positive integer.
  • performing pixel value prediction on the image block y by using the non-motion motion model and the combined motion information unit set j may include: when the merge motion In a case where the prediction direction in the information unit set j is different from the reference frame index corresponding to the motion vector in the first prediction direction, the merged motion information unit set j is subjected to scaling processing so that the merged motion information unit set j is The motion vector whose prediction direction is the first prediction direction is scaled to the same reference frame, and the pixel value prediction is performed on the image block y by using the non-translating motion model and the merged motion information unit set j after performing the scaling process, where The first prediction direction is forward or backward.
  • performing pixel value prediction on the image block y by using the non-translational motion model and the combined motion information unit set j may include: when the prediction direction in the combined motion information unit set j is a forward motion vector corresponding If the reference frame index is different, and the prediction direction in the merged motion information unit set j is different from the reference frame index corresponding to the backward motion vector, the merged motion information unit set j is subjected to scaling processing so that The prediction direction in the merged motion information unit set j is that the forward motion vector is scaled to the same reference frame and the backward motion vector in the merged motion information unit set j is scaled to the same reference frame And performing pixel value prediction on the image block y using the non-translational motion model and the merged motion information unit set j subjected to the scaling process.
  • performing pixel value prediction on the image block y by using the non-translating motion model and the combined motion information unit set j may include: The motion vector in the motion information unit set j performs motion estimation processing to obtain the motion estimation unit set j after the motion estimation processing, and the combined motion information unit set j using the non-motion motion model and the motion estimation processing Block y performs pixel value prediction.
  • performing pixel value prediction on the image block y by using the non-translating motion model and the merged motion information unit set j after performing the scaling process may include: Performing motion estimation processing on the motion vector in the merged motion information unit set j after the scaling processing to obtain the combined motion information unit set j after the motion estimation processing, and the combined motion information unit after the motion estimation processing and the motion estimation processing The set j performs pixel value prediction on the image block y.
  • the performing pixel value prediction on the image block y by using the non-translational motion model and the combined motion information unit set j includes: utilizing non-translation The motion model and the combined motion information unit set j calculate a motion vector of each pixel in the image block y, and determine the image block y by using the calculated motion vector of each pixel in the image block y. a predicted pixel value for each pixel in ;
  • the test finds that if the motion vector of each pixel block in the image block y is first calculated by using the non-translational motion model and the combined motion information unit set j, then each of the calculated image blocks y is used.
  • the motion vector of the pixel block determines the predicted pixel value of each pixel point of each pixel block in the image block y, since the calculation of the motion vector is performed by using the pixel block in the image block y as a granularity, which is advantageous for a large reduction Computational complexity.
  • FIG. 2-a is a schematic flowchart diagram of another image prediction method according to another embodiment of the present invention.
  • another image prediction method provided by another embodiment of the present invention may include:
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, and a lower left pixel sample of the image block x as an example.
  • the upper left pixel sample of the image block x is an upper left vertex of the image block x or a pixel block of the image block x including an upper left vertex of the image block x.
  • the lower left pixel sample of the image block x is the lower left vertex of the image block x or the pixel block of the image block x containing the lower left vertex of the image block x.
  • the upper right pixel sample of the image block x is the upper right vertex of the image block x or the pixel block of the image block x containing the upper right vertex of the image block x.
  • the central prime sample a1 of the image block x is a central pixel of the image block x or a pixel block of the image block x containing a central pixel of the image block x.
  • the size of the pixel block is, for example, 2*2, 1*2, 4*2, 4*4, or other sizes.
  • the candidate motion information unit set corresponding to each pixel sample includes at least one motion information unit of the candidate.
  • the pixel samples mentioned in the embodiments of the present invention may be pixel points or pixel blocks including at least two pixel points.
  • referring to the motion information unit in each embodiment of the present invention may include a motion vector in which the prediction direction is forward and/or a motion vector in which the prediction direction is backward. That is, one motion information unit may include one motion vector or two motion vectors that may include different prediction directions.
  • the prediction direction corresponding to the motion information unit is forward, it indicates that the motion information unit includes a motion vector whose prediction direction is forward, but does not include a motion vector whose prediction direction is backward. If the prediction direction corresponding to the motion information unit is backward, it indicates that the motion information unit includes a motion vector whose prediction direction is backward but does not include a motion vector whose prediction direction is forward. If the prediction direction corresponding to the motion information unit is unidirectional, it indicates that the motion information unit includes a motion vector in which the prediction direction is forward, but does not include a motion vector in which the prediction direction is backward, or indicates that the motion information unit includes a backward direction. The motion vector does not include the forward direction motion vector. Wherein, if the prediction direction corresponding to the motion information unit is bidirectional, the motion information unit includes the motion direction in which the prediction direction is forward and the prediction direction is backward. Sport vector.
  • K1 is an integer greater than one.
  • the K1 can be equal to 2, 3, 4 or other values.
  • the manner in which the candidate motion information unit sets of the K1 pixel samples are determined may be various.
  • the candidate motion information unit set S1 corresponding to the upper left pixel sample of the image block x may include motion information units of x1 pixel samples.
  • the x1 pixel samples include: a pixel sample Col-LT having the same position as the upper left pixel sample LT of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs.
  • At least one of the spatial adjacent image block C on the left side of the image block x, the spatially adjacent image block A on the upper left of the image block x, and the spatially adjacent image block B on the upper side of the image block x may be acquired first.
  • the motion information unit of the spatial neighboring image block B, the motion information unit of the spatial adjacent image block C of the left side of the image block x, and the motion of the spatially adjacent image block A of the upper left of the image block x are added to the candidate motion information unit set corresponding to the upper left pixel sample of the image block x, if the air space to the left of the image block x a motion information unit of the adjacent image block C, a motion information unit of the spatially adjacent image block A of the upper left of the image block x, and a portion of the motion information unit of the spatial adjacent image block B of the upper side of the image block x
  • the candidate motion information unit set S1 is further subjected to deduplication processing (the number of motion information units in the candidate motion information unit set S1 after deduplication processing may be 1 or 2), if the motion information unit of the pixel sample Col-LT having the same position as
  • the motion information unit of the pixel sample Col-LT having the same position as the upper left pixel sample LT of the image block x among the video frames adjacent to the video frame time domain to which the image block x belongs is different from Any one of the candidate motion information unit sets S1 after the reprocessing, and the video frame to which the image block x belongs a motion information unit of a pixel sample Col-LT having the same position as an upper left pixel sample LT of the image block x among the video frames adjacent to each other in the time domain is added to the candidate motion information unit set S1 after the deduplication processing, If the number of motion information units in the candidate motion information unit set S1 is still less than three at this time, zero motion information units are added to the candidate motion information unit set S1 until the candidate motion information unit set S1 The number of motion information units is equal to three.
  • the zero motion information unit added to the candidate motion information unit set S1 includes a zero motion vector whose prediction direction is forward but may not include a backward direction. Motion vector. If the video frame to which the image block x belongs is a backward predicted frame, the zero motion information unit added to the candidate motion information unit set S1 includes a zero motion vector whose prediction direction is backward but may not include a zero motion vector whose prediction direction is forward. .
  • the zero motion information unit added to the candidate motion information unit set S1 includes a zero motion vector whose prediction direction is forward and a zero motion vector whose prediction direction is backward.
  • the reference frame index corresponding to the motion vector added to the different zero motion information unit in the candidate motion information unit set S1 may be different, and the corresponding reference frame index may be, for example, 0, 1, 2, 3 or other values thereof.
  • the candidate motion information unit set S2 corresponding to the upper right pixel sample of the image block x may include motion information units of x2 image blocks.
  • the x2 image blocks may include: a pixel sample Col-RT having the same position as the upper right pixel sample RT of the image block x among the video frames adjacent to the video frame time domain to which the image block x belongs. At least one of the spatially adjacent image block E of the upper right of the image block x and the spatially adjacent image block D of the upper side of the image block x.
  • the motion information unit of the spatially adjacent image block E of the upper right of the image block x and the motion information unit of the spatial adjacent image block D of the upper side of the image block x may be acquired first, and the acquired image block x will be acquired.
  • the motion information unit of the upper right spatial image adjacent block E and the motion information unit of the spatial adjacent image block D of the upper side of the image block x are added to the candidate motion information unit set corresponding to the upper right pixel sample of the image block x
  • the candidate motion information is The unit set S2 performs deduplication processing (the number of motion information units in the candidate motion information unit set S2 after the de-duplication processing is 1), if adjacent to the video frame time domain to which the image block x belongs a pixel sample Col-RT of the same position as the upper right pixel sample RT
  • any one of the candidate motion information unit sets S2 that is reprocessed may be the same as the image block x of the video frame adjacent to the video frame time domain to which the image block x belongs.
  • the motion information unit of the pixel sample Col-RT in which the upper right pixel sample RT position is the same is added to the candidate motion information unit set S2 after the deduplication processing, if the motion information unit in the candidate motion information unit set S2 at this time If the number is still less than two, the zero motion information unit is further added to the candidate motion information unit set S2 until the number of motion information units in the candidate motion information unit set S2 is equal to two.
  • the zero motion information unit added to the candidate motion information unit set S2 includes the zero motion vector whose prediction direction is forward but may not include the backward direction. Motion vector. If the video frame to which the image block x belongs is a backward predicted frame, the zero motion information unit added to the candidate motion information unit set S2 includes a zero motion vector whose prediction direction is backward but may not include a zero motion vector whose prediction direction is forward. . In addition, if the video frame to which the image block x belongs is a bidirectional prediction frame, the zero motion information unit added to the candidate motion information unit set S2 includes a zero motion vector whose prediction direction is forward and a zero motion vector whose prediction direction is backward.
  • the reference frame index corresponding to the motion vector added to the different zero motion information unit in the candidate motion information unit set S2 may be different, and the corresponding reference frame index may be, for example, 0, 1, 2, 3 or other values thereof.
  • the candidate motion information unit set S3 corresponding to the lower left pixel sample of the image block x may include motion information units of x3 image blocks.
  • the x3 image blocks may include: a pixel sample Col-LB having the same position as the lower left pixel sample LB of the image block x, among the video frames adjacent to the video frame time domain to which the image block x belongs. At least one of the left-side spatial adjacent image block G of the image block x and the spatially adjacent image block F to the left of the image block x.
  • the motion information unit of the left-side spatial adjacent image block G of the image block x and the motion information unit of the spatial adjacent image block F of the left side of the image block x may be acquired first, and the acquired image block x may be acquired.
  • the motion information unit of the left-side spatial adjacent image block G and the spatial adjacent image of the left side of the image block x The motion information unit of the block F is added to the candidate motion information unit set S3 corresponding to the lower left pixel sample of the image block x, if the motion information unit of the left adjacent spatial image block G of the image block x and the image If the motion information unit of the spatial adjacent image block F on the left side of the block x is the same, the candidate motion information unit set S3 is subjected to deduplication processing (the motion in the candidate motion information unit set S3 after the deduplication processing)
  • the number of information units is 1), if the motion of the pixel sample Col-LB that is the same as the position of the lower left pixel sample LB of the image block x among the
  • any one of the candidate motion information unit sets S3 that is reprocessed may be the same as the image block x of the video frame adjacent to the video frame time domain to which the image block x belongs.
  • the motion information unit of the pixel sample Col-LB having the same lower left pixel sample LB position is added to the candidate motion information unit set S3 after the deduplication processing, if the motion information unit in the candidate motion information unit set S3 at this time If the number is still less than two, the zero motion information unit is further added to the candidate motion information unit set S3 until the number of motion information units in the candidate motion information unit set S3 is equal to two.
  • the zero motion information unit added to the candidate motion information unit set S3 includes the zero motion vector whose prediction direction is forward but may not include the backward direction. Motion vector. If the video frame to which the image block x belongs is a backward predicted frame, the zero motion information unit added to the candidate motion information unit set S3 includes a zero motion vector whose prediction direction is backward but may not include a zero motion vector whose prediction direction is forward. . In addition, if the video frame to which the image block x belongs is a bidirectional prediction frame, the zero motion information unit added to the candidate motion information unit set S3 includes a zero motion vector whose prediction direction is forward and a zero motion vector whose prediction direction is backward.
  • the reference frame index corresponding to the motion vector added to the different zero motion information unit in the candidate motion information unit set S3 may be different, and the corresponding reference frame index may be, for example, 0, 1, 2, 3 or other values thereof.
  • the two motion information units are different, and the motion information included in the two motion information units may be different, or the motion vectors included in the two motion information units may have different prediction directions, or The motion vectors included in the two motion information units have different reference frame indices.
  • the two motion information units are the same, and the motion vectors included in the two motion information units are the same, and the motion directions corresponding to the motion information units of the two motion information units are the same, and the two motion information units are The included motion vectors correspond to the same reference frame index.
  • a candidate motion information unit set of the corresponding pixel sample can be obtained in a similar manner.
  • the example shown in FIG. 2-d is exemplified by the fact that K1 is equal to 4.
  • the K1 pixel samples may include an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and a central pixel sample a1 of the image block x.
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x including the upper left vertex of the image block x;
  • the lower left pixel sample of the image block x a lower left vertex of the image block x or a pixel block in the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex or a location of the image block x a pixel block in the image block x that includes the upper right vertex of the image block x;
  • a central pixel sample a1 of the image block x is a central pixel point of the image block x or the image block x includes the A block of pixels at the center pixel of image block x.
  • each motion information unit included in each candidate merged motion information unit set in the N candidate combined motion information unit sets is respectively selected from candidate motion information corresponding to different pixel samples in the K1 pixel samples. At least part of the motion information element of the unit set that meets the constraint.
  • the set of N candidate combined motion information units are different from each other, and each set of candidate combined motion information units in the N candidate combined motion information unit sets includes K1 motion information units, and the N is a positive integer.
  • the number of motion information units included in the candidate motion information unit set S1, the candidate motion information unit set S2, and the candidate motion information unit set S3 is not limited to the above example, the number of initial candidate combined motion information unit sets is Not necessarily 12.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition. condition.
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the motion mode of the image block x indicated by the motion information unit in the candidate combined motion information unit set may be considered as a translational motion, and vice versa, the motion of the image block x indicated by the motion information unit in the candidate combined motion information unit is considered to be a non-translational motion, wherein the first prediction direction is forward or backward .
  • the candidate may be considered as the candidate.
  • the motion mode of the image block x indicated by the motion information unit in the merged motion information unit is a translational motion, and vice versa, the image block indicated by the motion information unit in the candidate combined motion information unit may be considered.
  • the mode of motion of x is non-translational motion.
  • the second condition may include that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction (if each motion of the candidate combined motion information unit is concentrated) If the prediction directions corresponding to the information units are different from each other, it may be considered that the certain candidate combined motion information unit set is invalid, and the certain candidate combined motion information unit set may be discarded. For example, when both motion information units include a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward, it indicates that the prediction directions corresponding to the two motion information units are the same.
  • the other motion information unit when one of the two motion information units includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward, the other motion information unit includes a motion vector in which the prediction direction is forward. Does not include a motion vector whose prediction direction is backward, or the other motion information unit includes a motion vector whose prediction direction is backward but does not include a motion in which the prediction direction is forward.
  • the vector can indicate that the prediction directions corresponding to the two motion information units are different.
  • one of the two motion information units when one of the two motion information units includes a motion vector whose prediction direction is a forward motion but does not include a motion vector whose backward direction is a backward direction, and another motion information unit includes a backward direction of the prediction direction.
  • the motion vector may indicate that the prediction directions corresponding to the two motion information units are different. For another example, when both motion information units include a motion vector whose prediction direction is forward, but the two motion information units do not include the motion vector whose prediction direction is backward, indicating that the prediction directions of the two motion information units are the same. . For another example, when both motion information units include a motion vector whose prediction direction is backward, but the two motion information units do not include a motion vector whose prediction direction is forward, indicating that the prediction directions of the two motion information units are the same. .
  • the third condition may include that the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same (if a candidate combined motion information unit) If the reference frame indexes corresponding to the respective motion information units are different from each other, the set of candidate combined motion information units may be considered to be invalid, and the certain candidate combined motion information unit set may be discarded. For example, when both motion information units include a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward, and the prediction direction in the two motion information units is a reference corresponding to the forward motion vector.
  • the frame index is the same, and the prediction direction in the two motion information units is the same as the reference frame index corresponding to the backward motion vector, which may indicate that the reference frame indexes corresponding to the two motion information units are the same.
  • the other motion information unit includes a motion vector in which the prediction direction is forward. Excluding the motion vector whose prediction direction is backward, or the other motion information unit includes the motion vector whose prediction direction is backward but does not include the motion vector whose prediction direction is forward, indicating the prediction direction corresponding to the two motion information units.
  • the reference frame indexes corresponding to the two motion information units may indicate that the reference frame indexes corresponding to the two motion information units are different.
  • the other motion information unit when one of the two motion information units includes a motion vector in which the prediction direction is a forward motion but does not include a motion vector in which the prediction direction is backward, the other motion information unit includes a motion in which the prediction direction is backward.
  • the vector, but not including the forward direction motion vector may indicate that the reference frame indices corresponding to the two motion information units are different.
  • the other The motion information unit includes a motion vector in which the prediction direction is forward, but does not include a motion vector in which the prediction direction is backward, and the prediction direction in the two motion information units is the same as the reference frame index corresponding to the forward motion vector, and can represent The reference frame indexes corresponding to the two motion information units are different.
  • the other motion information unit includes a motion in which the prediction direction is backward.
  • the vector does not include the motion vector of the forward direction
  • the reference frame in the two motion information units is the same as the reference frame index corresponding to the backward motion vector, and may represent the reference frame corresponding to the two motion information units.
  • the index is different.
  • the fourth condition includes that the absolute value of the difference of the horizontal components of two (eg, any two) motion information units in any one of the N candidate motion information unit sets is less than or equal to Horizontal component threshold.
  • the fourth condition includes two motions in which the prediction directions are the same in two (eg, any two) motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets.
  • the absolute value of the difference of the horizontal components of the vector is less than or equal to the horizontal component threshold. If the absolute value of the difference between the horizontal components of the two motion information units in a candidate combined motion information unit set is greater than the horizontal component threshold, the set of candidate motion information units may be considered illegal, and then may be discarded.
  • the horizontal component threshold example may be equal to 1/3 of the width of the image block x, 1/2 of the width of the image block x, 2/3 of the width of the image block x, or 3/4 of the width of the image block x or the like. size.
  • the fifth condition includes that the absolute value of the difference of the vertical components of two (eg, any two) motion information units in any one of the N candidate motion information unit sets is less than or equal to Vertical component threshold.
  • the fifth condition includes two motions in which the prediction directions are the same in two (eg, any two) motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets.
  • the absolute value of the difference of the vertical components of the vector is less than or equal to the vertical component threshold.
  • the above-mentioned vertical component threshold example may be equal to 1/3 of the height of the image block x, 1/2 of the height of the image block x, 2/3 of the height of the image block x, or 3/4 of the height of the image block x or the like. size.
  • the set of N candidate combined motion information units may, for example, also satisfy other unlisted conditions.
  • the initial candidate combined motion information element set is filtered by introducing at least one of the first condition, the second condition, the third condition, the fourth condition, and the fifth condition, from the initial candidate combined motion information unit set.
  • the N candidate motion information unit sets are filtered out, which is beneficial to early termination of some potentially invalid candidate motion information unit sets to participate in subsequent calculations, thereby facilitating the reduction of codec operation complexity.
  • the initial candidate combined motion information unit set may be filtered by using at least one of the first condition, the second condition, and the third condition, and the N01 is selected from the initial candidate combined motion information unit set.
  • the candidate merged motion information unit sets and then performs scaling processing on the N01 candidate combined motion information unit sets, and then uses at least one of the fourth condition and the fifth condition to extract from the N01 candidate combined motion information unit units subjected to the scaling processing.
  • N sets of candidate combined motion information unit sets are filtered out.
  • the fourth condition and the fifth condition may also not be referenced, but the initial candidate combined motion information element set is directly filtered by using at least one of the first condition, the second condition, and the third condition, from the initial candidate.
  • the combined motion information unit collectively filters out N candidate combined motion information unit sets.
  • the motion vector in the video codec reflects the distance that an object is offset in one direction (predictive direction) with respect to the same time (the same time frame corresponds to the same reference frame). Therefore, in the case that the motion information units of different pixel samples correspond to different prediction directions and/or correspond to different reference frame indexes, motion offset of each pixel/pixel block of the image block x with respect to a reference frame may not be directly obtained. And when the pixel samples correspond to the same prediction direction and correspond to the same reference frame index, the combined motion vector combination can be used to obtain the motion vector of each pixel/pixel block in the image block.
  • the candidate combined motion information unit set may be subjected to scaling processing.
  • performing scaling processing on the candidate combined motion information unit set may involve modifying, adding, and/or deleting motion vectors in one or more motion information units in the candidate combined motion information unit set.
  • the candidate combined motion may be performed
  • the set of information units performs scaling processing to cause the candidate combined motion information unit
  • the motion vector in which the concentrated prediction direction is the first prediction direction is scaled to the same reference frame, wherein the first prediction direction is forward or backward.
  • the prediction direction in the candidate combined motion information element set i (eg, the merged motion information unit set i) is different
  • the reference frame index corresponding to the forward motion vector is different
  • the prediction direction of the candidate combined motion information unit set is If the reference frame index corresponding to the backward motion vector is different, the candidate combined motion information unit set is subjected to scaling processing, so that the forward motion vector in the combined motion information unit set i is forward-oriented.
  • the motion vectors to the same reference frame and causing the prediction direction in the combined motion information element set i to be backward are scaled to the same reference frame.
  • the prediction direction corresponding to the motion information unit is bidirectional, and the motion information unit includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward.
  • the prediction direction corresponding to the motion information unit is forward, indicating that the motion information unit includes a motion vector in which the prediction direction is forward, but does not include a motion vector in which the prediction direction is backward.
  • the prediction direction corresponding to the motion information unit is backward, indicating that the motion information unit includes a motion vector whose prediction direction is backward but does not include a motion vector whose prediction direction is forward.
  • the corresponding information of the motion information units in the corresponding candidate combined motion information unit may also be different.
  • the prediction direction and/or the corresponding reference frame index are adjusted.
  • the prediction direction For the adjustment of the prediction direction, the following manner may be adopted. For example, if the prediction directions corresponding to the respective motion information units in the candidate combined motion information unit set z among the N candidate combined motion information unit sets are the same, the candidate combination may be maintained. The prediction direction corresponding to each motion information unit in the motion information unit set z does not change. In addition, if the prediction directions corresponding to the two motion information units in the candidate combined motion information unit set z are the same, the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z may be set to be the same as the above. The two motion vectors correspond to the same prediction direction.
  • the candidate combined motion information unit set z The prediction direction corresponding to the remaining motion information unit is the second prediction direction (second pre-pre The direction of the measurement is different from the first prediction direction, and the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z may be adjusted to the first prediction direction. That is, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are all the first prediction directions.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is bidirectional, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is forward, Adding a motion vector with a prediction direction to the backward motion vector in the remaining motion information unit in the candidate merge motion information unit set z (the newly added prediction direction is a backward motion vector and an original prediction in the corresponding motion information unit)
  • the direction of the forward motion vector is opposite, that is, the scaling process is performed according to 1:1, so that the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z becomes bidirectional. That is, the prediction directions corresponding to all motion information units in the candidate combined motion information unit set z are both bidirectional.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is bidirectional, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is backward, then Adding a prediction direction to the forward motion vector in the remaining motion information unit in the candidate merged motion information unit set z (the newly added prediction direction is a forward motion vector and an original prediction direction in the corresponding motion information unit)
  • the value of the backward motion vector is opposite, that is, the scaling process is performed according to 1:1, so that the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z becomes bidirectional. That is to say, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are both bidirectional.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is forward, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is bidirectional.
  • the motion vector in which the prediction direction is backward is deleted in the above-described remaining motion information unit in the candidate combined motion information unit set z.
  • the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z also becomes forward. That is, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are forward.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is backward, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is bidirectional,
  • the above remaining motion information in the candidate merged motion information unit set z The direction in which the prediction direction is the forward motion vector is deleted.
  • the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z also becomes backward. That is, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are all backward.
  • the adjustment of the reference frame index may be performed in the following manner. Specifically, if the reference frame index corresponding to each motion information unit in the candidate combined motion information unit set z among the N candidate combined motion information unit sets is the same, The reference frame index corresponding to each motion information unit in the candidate merged motion information unit set z remains unchanged.
  • the remaining in the candidate combined motion information unit set z may be
  • the reference frame index corresponding to the motion information unit is adjusted to be the same as the reference frame index corresponding to the two motion information units described above, and the candidate combined motion information unit set is subjected to scaling processing such that the combined motion information unit set i
  • the prediction direction in the direction is that the forward motion vector is scaled to the same reference frame and/or the motion vector in the backward direction in the combined motion information element set i is scaled to the same reference frame.
  • the method for scaling the motion vector MV d in the candidate merged motion information unit set z to the target reference frame may be as follows: assuming that the current video frame (such as the video of the image block x) frame) playback sequence number CurPoc, wherein the playing order number of the target reference frame is DesPoc, playback sequence number of the original reference frame motion vector MV d is SrcPoc, the motion vector MV d scaled to a target reference frame to obtain a motion vector For MV S.
  • the MV S can be obtained by the following formula:
  • MV S sign(ScaleFactor ⁇ MV d ) ⁇ abs(ScaleFactor ⁇ MV d )
  • the candidate combined motion information unit set z may be any one of the candidate motion information unit sets in the N candidate combined motion information unit sets, that is, for each candidate combined motion information unit set in the N candidate combined motion information unit sets,
  • the motion vector therein can be scaled according to the above exemplary manner.
  • the merged motion information unit set i may include: determining a combined motion information unit including K1 motion information units from among the N candidate combined motion information unit sets based on the identification of the merged motion information unit set i obtained from the video code stream Set i.
  • the method may further include: writing the identifier of the merged motion information unit set i Into the video stream.
  • the merged motion information unit set i may include: determining a combined motion information unit set i including K1 motion vectors from among the N candidate combined motion information unit sets based on a distortion or rate distortion cost.
  • the rate distortion cost corresponding to the merged motion information unit set i is less than or equal to the rate distortion cost corresponding to any one of the combined motion information unit sets except the combined motion information unit set i in the N candidate motion information unit sets. .
  • the distortion corresponding to the merged motion information unit set i is less than or equal to the distortion corresponding to any one of the combined motion information unit sets except the combined motion information unit set i in the N candidate motion information unit sets.
  • the rate-distortion cost corresponding to a certain candidate combined motion information unit set may be, for example, utilized.
  • the distortion corresponding to a certain candidate combined motion information unit set (for example, the combined motion information unit set i in the N candidate motion information unit sets) in the foregoing N candidate combined motion information unit sets, for example, may be an image block.
  • the original pixel value (such as image block x) and the pixel value prediction of the image block using the certain candidate combined motion information unit set (such as the merged motion information unit set i) Distortion between predicted pixel values of the image block (ie, distortion between the original pixel value of the image block and the predicted pixel value).
  • the original pixel value of the image block (such as image block x) and the pixel value prediction of the image block by using the certain candidate combined motion information unit set (for example, the combined motion information unit set i)
  • the resulting distortion between the predicted pixel values of the image block may be, for example, an original pixel value of the image block (eg, image block x) and a set of combined motion information units using the certain candidate (eg, a combined motion information unit set) i) a squared error sum (SSD) or absolute error sum (SAD) or error between the predicted pixel values of the image block resulting from pixel value prediction for the image block and or other distortion parameters capable of measuring distortion.
  • SSD squared error sum
  • SAD absolute error sum
  • n1 candidate combined motion information unit sets may be selected from the N candidate combined motion information unit sets, and n1 candidate combined motion information is obtained based on distortion or rate distortion cost.
  • the unit set determines a merged motion information unit set i including K1 motion information units.
  • the D(V) corresponding to any one of the candidate motion information unit sets in the n1 candidate combined motion information unit sets is less than or equal to any other than the n1 candidate combined motion information unit sets in the N candidate motion information unit sets.
  • a candidate merges the set of motion information units corresponding to D(V), where n1 is, for example, equal to 3, 4, 5, 6, or other values.
  • the identifier of the n1 candidate combined motion information unit set or the n1 candidate combined motion information unit set may be added to the candidate combined motion information unit set list, wherein if the N is less than or equal to n1, the N pieces may be used.
  • the identifier of the candidate combined motion information unit set or the N candidate combined motion information unit sets is added to the candidate combined motion information unit set list.
  • the candidate merged motion information unit set in the candidate merged motion information unit set list may be, for example, sorted in ascending or descending order according to the D(V) size.
  • the Euclidean distance parameter D(V) of the certain candidate combined motion information unit set can be calculated, for example, as follows: among them, The three motion vectors having the same prediction direction included in the set of candidate motion information units. It can be understood that, in the above example, the three motion information units included in the candidate combined motion information unit set are taken as an example, and the case of the other number of motion information units included in the candidate combined motion information unit set may be deduced.
  • the Euclidean distance parameter D(V) of the certain candidate combined motion information unit set can be calculated, for example, as follows: with The prediction direction included in the set of candidate motion information units is a forward three motion vectors. With The prediction direction included in the set of candidate motion information units is a backward three motion vectors. It can be understood that the above example is exemplified by three motion information units included in the candidate combined motion information unit set, and the other possible motion information units included in the candidate combined motion information unit set may be deduced.
  • the size of the image block x is S 1 ⁇ S 2 , and the S 1 is equal to or not equal to S 2 .
  • Figure 2-e shows the coordinates of the four vertices of image block x.
  • the motion vector of each pixel in the image block x can be calculated.
  • FIG. 2-e shows an affine transformation of pixels with coordinates (x, y)
  • FIG. 2-f shows coordinates (0, Affine transformation of three known pixel points of 0), (S 1 , 0) and (0, S 2 ).
  • the motion vector of the pixel with the coordinate (x, y) can be calculated based on the following formula:
  • the affine motion model is mainly described as an example of the non-translating motion model, and the motion vector calculation system based on the other non-motion motion models and the K pixel samples is described.
  • the prediction residual of the image block x may be obtained by using the original pixel value of the image block x and the image block x predicted pixel value obtained by performing pixel value prediction, and the prediction of the image block x may be performed.
  • the residual is written to the video stream.
  • bilinear interpolation is used for chrominance components.
  • the interpolation of the luminance component is shown in Figure 2-h.
  • a, b, c and d are 1/4 precision pixel points, which can be obtained by HEVC interpolation filter; the horizontal direction is obtained according to the motion vector of each pixel
  • bilinear interpolation can be performed, and S is the accuracy of bilinear interpolation.
  • the forward prediction value of the image block x and the image block x obtained in the backward reference frame are respectively obtained in the forward reference frame according to the above method.
  • the backward predicted value is then weighted averaged for the forward predicted value and the backward predicted value of the image block x to obtain the predicted pixel value of the image block x.
  • the storage efficiency of motion vectors is improved at the same time.
  • the current image block x may be stored by the minimum motion vector storage unit as follows (assuming that the minimum motion vector storage unit is a motion vector storing 4 ⁇ 4 image blocks): first store the current according to FIG. 2-i The motion vector of the 4x4 block at the 3 pixel samples of the prediction unit with Use the upper left corner Fill in other minimum motion vector storage units.
  • the motion vector of the lower right pixel sample of the image block x and the motion vector of the central pixel sample of the image block x can be calculated using the following formula.
  • motion vectors of other pixel samples except the top-angle pixel samples in the image block x may be stored as
  • the image block x is predicted by using the non-translational motion model and the combined motion information unit set i, and the motion information unit set i is merged from the N conditions that satisfy the condition.
  • Each of the N candidate motion information units is selected from at least part of the motion information of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples.
  • the unit because the selection range of the combined motion information unit set i is relatively small, discarding a motion of filtering out K1 pixel samples by a large number of calculations in all possible candidate motion information unit sets of K1 pixel samples used in the conventional technology
  • the mechanism of information unit is beneficial to improve coding efficiency, and it is also beneficial to reduce the computational complexity of image prediction based on non-translational motion model, which makes it possible to introduce non-translational motion model into video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • FIG. 3 is a schematic flowchart diagram of a video encoding method according to another embodiment of the present invention.
  • a video encoding method provided by another embodiment of the present invention may include:
  • the video encoding device determines K1 pixel samples in the image block x.
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, and a lower left pixel sample of the image block x as an example.
  • the upper left pixel sample of the image block x is an upper left vertex of the image block x or a pixel block of the image block x including an upper left vertex of the image block x.
  • the lower left pixel sample of the image block x is the lower left vertex of the image block x or the pixel block of the image block x containing the lower left vertex of the image block x.
  • the upper right pixel sample of the image block x is the upper right vertex of the image block x or the pixel block of the image block x containing the upper right vertex of the image block x.
  • the video encoding apparatus determines a candidate motion information unit set corresponding to each of the K1 pixel samples.
  • the candidate motion information unit set corresponding to each pixel sample includes at least one motion information unit of the candidate.
  • referring to the motion information unit in each embodiment of the present invention may include a motion vector in which the prediction direction is forward and/or a motion vector in which the prediction direction is backward. That is, one motion information unit may include one motion vector or two motion vectors that may include different prediction directions.
  • K1 is an integer greater than one.
  • the K1 can be equal to 2, 3, 4 or other values.
  • the two motion information units are different, and the motion information included in the two motion information units is different, or the motion directions corresponding to the motion information units of the two motion information units are different, or the two motion information units are The included motion vector corresponds to a different reference frame index.
  • the two motion information units are the same, and the motion vectors included in the two motion information units are the same, and the motion directions corresponding to the motion information units of the two motion information units are the same, and the two motion information units are The included motion vectors correspond to the same reference frame index.
  • the determining manner of the corresponding candidate motion information unit set of the K1 pixel samples may be various.
  • a set of candidate motion information units corresponding to each of the K1 pixel samples may be determined according to a related example in step 202.
  • the video encoding apparatus determines, according to the set of candidate motion information units corresponding to each of the K1 pixel samples, a set of N candidate combined motion information units.
  • Each motion information unit included in each candidate merged motion information unit set in the N candidate combined motion information unit sets is respectively selected from candidate motion information corresponding to different pixel samples in the K1 pixel samples. At least part of the motion information element of the unit set that meets the constraint.
  • the set of N candidate combined motion information units are different from each other, and each set of candidate combined motion information units in the N candidate combined motion information unit sets includes K1 motion information units, and the N is a positive integer.
  • at least one of the first condition, the second condition, and the third condition may be utilized to filter the set of N candidate combined motion information units from the 12 initial candidate combined motion information unit sets.
  • the number of motion information units included in the candidate motion information unit set S1, the candidate motion information unit set S2, and the candidate motion information unit set S3 is not limited to the above example, the number of initial candidate combined motion information unit sets is Not necessarily 12.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition. condition.
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the second condition may include that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction (if each motion of the candidate combined motion information unit is concentrated) If the prediction directions corresponding to the information units are different from each other, it may be considered that the certain candidate combined motion information unit set is invalid, and the certain candidate combined motion information unit set may be discarded.
  • the third condition may include that the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same (if a candidate combined motion information unit) If the reference frame indexes corresponding to the respective motion information units are different from each other, the set of candidate combined motion information units may be considered to be invalid, and the certain candidate combined motion information unit set may be discarded.
  • the fourth condition includes that the absolute value of the difference of the horizontal components of two (eg, any two) motion information units in any one of the N candidate motion information unit sets is less than or equal to Horizontal component threshold.
  • the horizontal component threshold example may be equal to 1/3 of the width of the image block x, 1/2 of the width of the image block x, 2/3 of the width of the image block x, or 3/4 of the width of the image block x or the like. size.
  • the fifth condition includes a vertical component of two (eg, any two) motion information units of any one of the candidate motion information unit sets in the N candidate combined motion information unit sets
  • the absolute value of the difference is less than or equal to the vertical component threshold.
  • the above-mentioned vertical component threshold example may be equal to 1/3 of the height of the image block x, 1/2 of the height of the image block x, 2/3 of the height of the image block x, or 3/4 of the height of the image block x or the like. size.
  • the set of N candidate combined motion information units may, for example, also satisfy other unlisted conditions.
  • the initial candidate combined motion information element set is filtered by introducing at least one of the first condition, the second condition, the third condition, the fourth condition, and the fifth condition, from the initial candidate combined motion information unit set.
  • the N candidate motion information unit sets are filtered out, which is beneficial to early termination of some potentially invalid candidate motion information unit sets to participate in subsequent calculations, thereby facilitating the reduction of codec operation complexity.
  • the initial candidate combined motion information unit set may be filtered by using at least one of the first condition, the second condition, and the third condition, and the N01 is selected from the initial candidate combined motion information unit set.
  • the candidate merged motion information unit sets and then performs scaling processing on the N01 candidate combined motion information unit sets, and then uses at least one of the fourth condition and the fifth condition to extract from the N01 candidate combined motion information unit units subjected to the scaling processing.
  • N sets of candidate combined motion information unit sets are filtered out.
  • the fourth condition and the fifth condition may also not be referenced, but the initial candidate combined motion information element set is directly filtered by using at least one of the first condition, the second condition, and the third condition, from the initial candidate.
  • the combined motion information unit collectively filters out N candidate combined motion information unit sets.
  • the motion vector in the video codec reflects the distance that an object is offset in one direction (predictive direction) with respect to the same time (the same time frame corresponds to the same reference frame). Therefore, in the case that the motion information units of different pixel samples correspond to different prediction directions and/or correspond to different reference frame indexes, motion offset of each pixel/pixel block of the image block x with respect to a reference frame may not be directly obtained. And when the pixel samples correspond to the same prediction direction and correspond to the same reference frame index, the combined motion vector combination can be used to obtain the motion vector of each pixel/pixel block in the image block.
  • the candidate combined motion information unit set may be subjected to scaling processing.
  • performing scaling processing on the candidate combined motion information unit set may involve motion vectors in one or more motion information units in the candidate combined motion information unit set. Modifications, additions, and/or deletions.
  • the candidate combined motion may be performed
  • the information unit set performs a scaling process such that the motion vector of the candidate combined motion information unit set in the prediction direction of the first prediction direction is scaled to the same reference frame, wherein the first prediction direction is forward or backward.
  • the prediction direction in the candidate combined motion information element set i (eg, the merged motion information unit set i) is different
  • the reference frame index corresponding to the forward motion vector is different
  • the prediction direction of the candidate combined motion information unit set is If the reference frame index corresponding to the backward motion vector is different, the candidate combined motion information unit set is subjected to scaling processing, so that the forward motion vector in the combined motion information unit set i is forward-oriented.
  • the motion vectors to the same reference frame and causing the prediction direction in the combined motion information element set i to be backward are scaled to the same reference frame.
  • the prediction direction corresponding to the motion information unit is bidirectional, and the motion information unit includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward.
  • the prediction direction corresponding to the motion information unit is forward, indicating that the motion information unit includes a motion vector in which the prediction direction is forward, but does not include a motion vector in which the prediction direction is backward.
  • the prediction direction corresponding to the motion information unit is backward, indicating that the motion information unit includes a motion vector whose prediction direction is backward but does not include a motion vector whose prediction direction is forward.
  • the corresponding information of the motion information units in the corresponding candidate combined motion information unit may also be different.
  • the prediction direction and/or the corresponding reference frame index are adjusted.
  • the prediction direction For the adjustment of the prediction direction, the following manner may be adopted. For example, if the prediction directions corresponding to the respective motion information units in the candidate combined motion information unit set z among the N candidate combined motion information unit sets are the same, the candidate combination may be maintained. The prediction direction corresponding to each motion information unit in the motion information unit set z does not change. In addition, if the prediction directions corresponding to the two motion information units in the candidate combined motion information unit set z are the same, the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z may be set to be the same as the above. Prediction direction corresponding to two motion vectors the same.
  • the candidate combined motion information unit set z The prediction direction corresponding to the remaining motion information unit is the second prediction direction (the second prediction direction is different from the first prediction direction), and the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z may be adjusted.
  • the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are all the first prediction directions.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is bidirectional, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is forward, Adding a motion vector with a prediction direction to the backward motion vector in the remaining motion information unit in the candidate merge motion information unit set z (the newly added prediction direction is a backward motion vector and an original prediction in the corresponding motion information unit)
  • the direction of the forward motion vector is opposite, that is, the scaling process is performed according to 1:1, so that the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z becomes bidirectional. That is, the prediction directions corresponding to all motion information units in the candidate combined motion information unit set z are both bidirectional.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is bidirectional, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is backward, then Adding a prediction direction to the forward motion vector in the remaining motion information unit in the candidate merged motion information unit set z (the newly added prediction direction is a forward motion vector and an original prediction direction in the corresponding motion information unit)
  • the value of the backward motion vector is opposite, that is, the scaling process is performed according to 1:1, so that the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z becomes bidirectional. That is to say, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are both bidirectional.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is forward, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is bidirectional.
  • the motion vector in which the prediction direction is backward is deleted in the above-described remaining motion information unit in the candidate combined motion information unit set z.
  • the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z also becomes forward. That is, candidate combined motion information
  • the prediction directions corresponding to all motion information units in the unit set z are forward.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is backward, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is bidirectional.
  • the forward motion vector whose prediction direction is forward is deleted in the above remaining motion information unit in the candidate merged motion information unit set z.
  • the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z also becomes backward. That is, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are all backward.
  • the adjustment of the reference frame index may be performed in the following manner. Specifically, if the reference frame index corresponding to each motion information unit in the candidate combined motion information unit set z among the N candidate combined motion information unit sets is the same, The reference frame index corresponding to each motion information unit in the candidate merged motion information unit set z remains unchanged.
  • the remaining in the candidate combined motion information unit set z may be
  • the reference frame index corresponding to the motion information unit is adjusted to be the same as the reference frame index corresponding to the two motion information units described above, and the candidate combined motion information unit set is subjected to scaling processing such that the combined motion information unit set i
  • the prediction direction in the direction is that the forward motion vector is scaled to the same reference frame and/or the motion vector in the backward direction in the combined motion information element set i is scaled to the same reference frame.
  • the candidate combined motion information unit set z may be any one of the candidate motion information unit sets in the N candidate combined motion information unit sets, that is, for each candidate combined motion information unit set in the N candidate combined motion information unit sets,
  • the motion vector therein can be scaled according to the above exemplary manner.
  • the video encoding device determines, from the set of N candidate combined motion information units, a combined motion information unit set i including K1 motion information units based on a distortion or rate distortion cost.
  • the rate distortion cost corresponding to the merged motion information unit set i is less than or equal to the rate distortion cost corresponding to any one of the combined motion information unit sets except the combined motion information unit set i in the N candidate motion information unit sets. .
  • the distortion corresponding to the combined motion information unit set i is less than or equal to any one of the N candidate motion information unit sets except the combined motion information unit set i.
  • the distortion corresponding to the metaset is less than or equal to any one of the N candidate motion information unit sets except the combined motion information unit set i.
  • the rate-distortion cost corresponding to a certain candidate combined motion information unit set may be, for example, utilized.
  • the distortion corresponding to a certain candidate combined motion information unit set (for example, the combined motion information unit set i in the N candidate motion information unit sets) in the foregoing N candidate combined motion information unit sets, for example, may be an image block.
  • Distortion ie, distortion between the original pixel value of the image block and the predicted pixel value.
  • the original pixel value of the image block (such as image block x) and the pixel value prediction of the image block by using the certain candidate combined motion information unit set (for example, the combined motion information unit set i)
  • the resulting distortion between the predicted pixel values of the image block may be, for example, an original pixel value of the image block (eg, image block x) and a set of combined motion information units using the certain candidate (eg, a combined motion information unit set) i) a squared error sum (SSD) or absolute error sum (SAD) or error between the predicted pixel values of the image block resulting from pixel value prediction for the image block and or other distortion parameters capable of measuring distortion.
  • SSD squared error sum
  • SAD absolute error sum
  • n1 candidate combined motion information unit sets may be selected from the N candidate combined motion information unit sets, and n1 candidate combined motion information is obtained based on distortion or rate distortion cost.
  • the unit set determines a merged motion information unit set i including K1 motion information units.
  • the D(V) corresponding to any one of the candidate motion information unit sets in the n1 candidate combined motion information unit sets is less than or equal to any other than the n1 candidate combined motion information unit sets in the N candidate motion information unit sets.
  • a candidate merges the set of motion information units corresponding to D(V), where n1 is, for example, equal to 3, 4, 5, 6, or other values.
  • the identifier of the n1 candidate combined motion information unit set or the n1 candidate combined motion information unit set may be added to the candidate combined motion information unit set list, where if the above N is less than or If it is equal to n1, the identifiers of the N candidate motion information unit sets or the N candidate motion information unit sets may be added to the candidate combined motion information unit set list.
  • the candidate merged motion information unit set in the candidate merged motion information unit set list may be, for example, sorted in ascending or descending order according to the D(V) size.
  • the Euclidean distance parameter D(V) of the certain candidate combined motion information unit set can be calculated, for example, as follows: among them, The three motion vectors having the same prediction direction included in the set of candidate motion information units. It can be understood that, in the above example, the three motion information units included in the candidate combined motion information unit set are taken as an example, and the case of the other number of motion information units included in the candidate combined motion information unit set may be deduced.
  • the Euclidean distance parameter D(V) of the certain candidate combined motion information unit set can be calculated, for example, as follows: with The prediction direction included in the set of candidate motion information units is a forward three motion vectors. With The prediction direction included in the set of candidate motion information units is a backward three motion vectors. It can be understood that the above example is exemplified by three motion information units included in the candidate combined motion information unit set, and the other possible motion information units included in the candidate combined motion information unit set may be deduced.
  • the video encoding apparatus performs motion estimation processing on the motion vector in the combined motion information unit set i to obtain the combined motion information unit set i after the motion estimation processing, and the video encoding apparatus uses the non-translation motion model and the motion estimation processing.
  • the merged motion information unit set i performs motion vector prediction on the image block x.
  • the video encoding apparatus performs pixel value prediction on the image block x based on the calculated motion vector of each pixel point or each pixel block of the image block x.
  • the video encoding apparatus obtains a prediction residual of the image block x by using the original pixel value of the image block x and the image block x predicted pixel value obtained by performing pixel value prediction on the image block x.
  • the video encoding apparatus writes the prediction residual of the image block x into the video code stream.
  • the image prediction method is applied to In the case of a video encoding process, the method may further include writing the identification of the combined motion information element set i to the video code stream. For example, when the N is greater than 1, the identifier of the combined motion information unit set i may be written into the video code stream. Since the combined motion information unit set i is directly and uniquely determinable when N is equal to 1, the encoding end writes the identifier of the combined motion information unit set i to the video code even when the N is equal to 1. Stream, the decoder can still determine the combined motion information unit set i
  • the forward prediction value of the image block x and the image block x obtained in the backward reference frame are respectively obtained in the forward reference frame according to the above method.
  • the backward predicted value is then weighted averaged for the forward predicted value and the backward predicted value of the image block x to obtain the predicted pixel value of the image block x.
  • pixel value prediction is performed on the image block x by using a non-motion motion model and a combined motion information unit set i, and the motion information unit set i is merged from N candidates satisfying the condition.
  • Each of the N candidate motion information units is selected from at least a part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples. Since the selection range of the merged motion information unit set i is relatively small, a motion information of K1 pixel samples is filtered out by a large number of calculations in all possible candidate motion information unit sets of the K1 pixel samples used in the conventional technique.
  • the mechanism of the unit is beneficial to improve the coding efficiency, and it is also beneficial to reduce the computational complexity of image prediction based on the non-translational motion model, which makes it possible to introduce the non-translational motion model into the video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • FIG. 4 is a schematic flowchart diagram of a video encoding method according to another embodiment of the present invention.
  • a video encoding method provided by another embodiment of the present invention may include:
  • the video decoding apparatus determines K1 pixel samples in the image block x.
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, and a lower left pixel sample of the image block x as an example.
  • the upper left pixel sample of the image block x is an upper left vertex of the image block x or a pixel block of the image block x including an upper left vertex of the image block x.
  • the lower left pixel sample of the image block x It is a lower left vertex of the image block x or a pixel block in the image block x containing the lower left vertex of the image block x.
  • the upper right pixel sample of the image block x is the upper right vertex of the image block x or the pixel block of the image block x containing the upper right vertex of the image block x.
  • the size of the pixel block is, for example, 2*2, 1*2, 4*2, 4*4, or other sizes.
  • the video decoding device determines a candidate motion information unit set corresponding to each of the K1 pixel samples.
  • the candidate motion information unit set corresponding to each pixel sample includes at least one motion information unit of the candidate.
  • referring to the motion information unit in each embodiment of the present invention may include a motion vector in which the prediction direction is forward and/or a motion vector in which the prediction direction is backward. That is, one motion information unit may include one motion vector or two motion vectors that may include different prediction directions.
  • the prediction direction corresponding to the motion information unit is forward, it indicates that the motion information unit includes a motion vector whose prediction direction is forward, but does not include a motion vector whose prediction direction is backward. If the prediction direction corresponding to the motion information unit is backward, it indicates that the motion information unit includes a motion vector whose prediction direction is backward but does not include a motion vector whose prediction direction is forward. If the prediction direction corresponding to the motion information unit is unidirectional, it indicates that the motion information unit includes a motion vector in which the prediction direction is forward, but does not include a motion vector in which the prediction direction is backward, or indicates that the motion information unit includes a backward direction. The motion vector does not include the forward direction motion vector. Wherein, if the prediction direction corresponding to the motion information unit is bidirectional, the motion information unit includes a motion vector in which the prediction direction is forward and a motion vector in which the prediction direction is backward.
  • K1 is an integer greater than one.
  • the K1 can be equal to 2, 3, 4 or other values.
  • the determining manner of the corresponding candidate motion information unit set of the K1 pixel samples may be various.
  • a set of candidate motion information units corresponding to each of the K1 pixel samples may be determined according to a related example in step 202.
  • the video decoding apparatus determines, according to the set of candidate motion information units corresponding to each of the K1 pixel samples, a set of N candidate combined motion information units. Wherein each motion letter included in each candidate merged motion information unit set in the N candidate combined motion information unit sets The information unit is respectively selected from at least part of the motion information unit that meets the constraint condition in the candidate motion information unit set corresponding to the different pixel samples in the K1 pixel samples.
  • the set of N candidate combined motion information units are different from each other, and each set of candidate combined motion information units in the N candidate combined motion information unit sets includes K1 motion information units, and the N is a positive integer.
  • the number of motion information units included in the candidate motion information unit set S1, the candidate motion information unit set S2, and the candidate motion information unit set S3 is not limited to the above example, the number of initial candidate combined motion information unit sets is Not necessarily 12.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition. condition.
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the second condition may include that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction (if each motion of the candidate combined motion information unit is concentrated) If the prediction directions corresponding to the information units are different from each other, it may be considered that the certain candidate combined motion information unit set is invalid, and the certain candidate combined motion information unit set may be discarded.
  • the third condition may include that the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same (if a candidate combined motion information unit) If the reference frame indexes corresponding to the motion information units in the set are different from each other, the set of candidate motion information units may be considered as invalid, and the certain frame may be discarded.
  • Candidate merged motion information unit set If the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same (if a candidate combined motion information unit) If the reference frame indexes corresponding to the motion information units in the set are different from each other, the set of candidate motion information units may be considered as invalid, and the certain frame may be discarded.
  • Candidate merged motion information unit set may include that the reference frame index corresponding to the at least two motion information units in any one of the N candidate motion information unit sets is the same (if a candidate combined motion information unit) If the reference frame indexes corresponding to the motion information units in the set are
  • the fourth condition includes that the absolute value of the difference of the horizontal components of two (eg, any two) motion information units in any one of the N candidate motion information unit sets is less than or equal to Horizontal component threshold.
  • the horizontal component threshold example may be equal to 1/3 of the width of the image block x, 1/2 of the width of the image block x, 2/3 of the width of the image block x, or 3/4 of the width of the image block x or the like. size.
  • the fifth condition includes that the absolute value of the difference of the vertical components of two (eg, any two) motion information units in any one of the N candidate motion information unit sets is less than or equal to Vertical component threshold.
  • the above-mentioned vertical component threshold example may be equal to 1/3 of the height of the image block x, 1/2 of the height of the image block x, 2/3 of the height of the image block x, or 3/4 of the height of the image block x or the like. size.
  • the set of N candidate combined motion information units may, for example, also satisfy other unlisted conditions.
  • the initial candidate combined motion information element set is filtered by introducing at least one of the first condition, the second condition, the third condition, the fourth condition, and the fifth condition, from the initial candidate combined motion information unit set.
  • the N candidate motion information unit sets are filtered out, which is beneficial to early termination of some potentially invalid candidate motion information unit sets to participate in subsequent calculations, thereby facilitating the reduction of codec operation complexity.
  • the initial candidate combined motion information unit set may be filtered by using at least one of the first condition, the second condition, and the third condition, and the N01 is selected from the initial candidate combined motion information unit set.
  • the candidate merged motion information unit sets and then performs scaling processing on the N01 candidate combined motion information unit sets, and then uses at least one of the fourth condition and the fifth condition to extract from the N01 candidate combined motion information unit units subjected to the scaling processing.
  • N sets of candidate combined motion information unit sets are filtered out.
  • the fourth condition and the fifth condition may also not be referenced, but the initial candidate combined motion information element set is directly filtered by using at least one of the first condition, the second condition, and the third condition, from the initial candidate.
  • the combined motion information unit collectively filters out N candidate combined motion information unit sets.
  • the motion vector in the video codec reflects the distance that an object is offset in one direction (predictive direction) with respect to the same time (the same time frame corresponds to the same reference frame).
  • motion offset of each pixel/pixel block of the image block x with respect to a reference frame may not be directly obtained.
  • the combined motion vector combination can be used to obtain the motion vector of each pixel/pixel block in the image block.
  • the candidate combined motion information unit set may be subjected to scaling processing.
  • performing scaling processing on the candidate combined motion information unit set may involve modifying, adding, and/or deleting motion vectors in one or more motion information units in the candidate combined motion information unit set.
  • the candidate combined motion may be performed
  • the information unit set performs a scaling process such that the motion vector of the candidate combined motion information unit set in the prediction direction of the first prediction direction is scaled to the same reference frame, wherein the first prediction direction is forward or backward.
  • the prediction direction in the candidate combined motion information element set i (eg, the merged motion information unit set i) is different
  • the reference frame index corresponding to the forward motion vector is different
  • the prediction direction of the candidate combined motion information unit set is If the reference frame index corresponding to the backward motion vector is different, the candidate combined motion information unit set is subjected to scaling processing, so that the forward motion vector in the combined motion information unit set i is forward-oriented.
  • the motion vectors to the same reference frame and causing the prediction direction in the combined motion information element set i to be backward are scaled to the same reference frame.
  • the corresponding information of the motion information units in the corresponding candidate combined motion information unit may also be different.
  • the prediction direction and/or the corresponding reference frame index are adjusted.
  • the prediction direction For the adjustment of the prediction direction, the following manner may be adopted. For example, if the prediction directions corresponding to the respective motion information units in the candidate combined motion information unit set z among the N candidate combined motion information unit sets are the same, the candidate combination may be maintained. The prediction direction corresponding to each motion information unit in the motion information unit set z does not change. In addition, if two of the candidate motion information unit sets z are in the motion letter If the prediction directions corresponding to the information units are the same, the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z may be set to be the same as the prediction direction corresponding to the two motion vectors.
  • the candidate combined motion information unit set z The prediction direction corresponding to the remaining motion information unit is the second prediction direction (the second prediction direction is different from the first prediction direction), and the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z may be adjusted.
  • the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are all the first prediction directions.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is bidirectional, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is forward, Adding a motion vector with a prediction direction to the backward motion vector in the remaining motion information unit in the candidate merge motion information unit set z (the newly added prediction direction is a backward motion vector and an original prediction in the corresponding motion information unit)
  • the direction of the forward motion vector is opposite, that is, the scaling process is performed according to 1:1, so that the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z becomes bidirectional. That is, the prediction directions corresponding to all motion information units in the candidate combined motion information unit set z are both bidirectional.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is bidirectional, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is backward, then Adding a prediction direction to the forward motion vector in the remaining motion information unit in the candidate merged motion information unit set z (the newly added prediction direction is a forward motion vector and an original prediction direction in the corresponding motion information unit)
  • the value of the backward motion vector is opposite, that is, the scaling process is performed according to 1:1, so that the prediction direction corresponding to the remaining motion information unit in the candidate combined motion information unit set z becomes bidirectional. That is to say, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are both bidirectional.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is forward, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is bidirectional,
  • the above remaining motion information in the candidate merged motion information unit set z The motion direction in which the prediction direction is backward is deleted in the unit.
  • the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z also becomes forward. That is, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are forward.
  • the prediction direction corresponding to two motion information units in the candidate combined motion information unit set z is backward, and the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z is bidirectional.
  • the forward motion vector whose prediction direction is forward is deleted in the above remaining motion information unit in the candidate merged motion information unit set z.
  • the prediction direction corresponding to the remaining motion information units in the candidate combined motion information unit set z also becomes backward. That is, the prediction directions corresponding to all the motion information units in the candidate combined motion information unit set z are all backward.
  • the adjustment of the reference frame index may be performed in the following manner. Specifically, if the reference frame index corresponding to each motion information unit in the candidate combined motion information unit set z among the N candidate combined motion information unit sets is the same, The reference frame index corresponding to each motion information unit in the candidate merged motion information unit set z remains unchanged.
  • the remaining in the candidate combined motion information unit set z may be
  • the reference frame index corresponding to the motion information unit is adjusted to be the same as the reference frame index corresponding to the two motion information units described above, and the candidate combined motion information unit set is subjected to scaling processing such that the combined motion information unit set i
  • the prediction direction in the direction is that the forward motion vector is scaled to the same reference frame and/or the motion vector in the backward direction in the combined motion information element set i is scaled to the same reference frame.
  • the candidate combined motion information unit set z may be any one of the candidate motion information unit sets in the N candidate combined motion information unit sets, that is, for each candidate combined motion information unit set in the N candidate combined motion information unit sets,
  • the motion vector therein can be scaled according to the above exemplary manner.
  • the video decoding apparatus performs decoding processing on the video code stream to obtain an identifier of the combined motion information unit set i and a prediction residual of the image block x, and based on the identifier of the combined motion information unit set i, from the N candidate combined motion information unit sets. Among them, a combined motion information unit set i including K1 motion information units is determined.
  • the video decoding apparatus performs motion estimation on the motion vector in the merged motion information unit set i
  • the processing is performed to obtain the combined motion information unit set i after the motion estimation process, and the video decoding apparatus performs motion vector prediction on the image block x using the non-translating motion model and the combined motion information unit set i after the motion estimation processing.
  • the video decoding device predicts the pixel value of the image block x obtained by performing pixel value prediction on the image block x based on the calculated motion vector of each pixel point or each pixel block of the image block x.
  • the video decoding apparatus reconstructs the image block x by using the image block x predicted pixel value and the prediction residual of the image block x.
  • pixel value prediction is performed on the image block x by using a non-translational motion model and a combined motion information unit set i, and the motion information unit set i is merged from N candidates satisfying the condition.
  • Each of the N candidate motion information units is selected from at least a part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples. Since the selection range of the merged motion information unit set i is relatively small, a motion information of K1 pixel samples is filtered out by a large number of calculations in all possible candidate motion information unit sets of the K1 pixel samples used in the conventional technique.
  • the mechanism of the unit is beneficial to improve the decoding efficiency, and it is also beneficial to reduce the computational complexity of image prediction based on the non-translational motion model, which makes it possible to introduce the non-translational motion model into the video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • an affine motion model is a motion model that satisfies the following model:
  • the rotational motion model can be as follows:
  • ⁇ is the angle of rotation and a and b are parameters.
  • the perspective motion model can be as follows:
  • the shear motion model can be as follows:
  • the other is a constant parameter.
  • the scaled motion model can be as follows:
  • ⁇ x , ⁇ y , ⁇ x and ⁇ y are constant parameters.
  • the secondary motion model can be as follows:
  • u and v represent the original coordinate points
  • x and y represent the transformed coordinates
  • others are constant parameters.
  • the bilinear motion model can be as follows:
  • a motion vector of any point in the object can be obtained based on the bilinear motion model, and the function represents as follows
  • a motion vector affine motion model in the case of multiple reference frames is established, which is advantageous for overcoming the defect that the prior art cannot be effectively used in the case of multiple reference frames; and is selected by introducing a screening condition game.
  • Some invalid combined motion information unit sets which in turn helps to improve coding efficiency.
  • Table 1 shows the coding efficiency in the LDP mode. Among them, under standard test conditions, the average can save 1.6% of the code rate, and can save up to 4.7% of the code rate.
  • an embodiment of the present invention further provides an image prediction apparatus 500, which may include:
  • a first determining unit 510 configured to determine K1 pixel samples in the image block x, and determine a candidate motion information unit set corresponding to each of the K1 pixel samples;
  • the set of candidate motion information units includes at least one motion information unit of the candidate, and the K1 is an integer greater than or equal to 2.
  • a second determining unit 520 configured to determine a combined motion information unit set i including K1 motion information units, where each motion information unit in the combined motion information unit set i is selected from the K1 pixel samples respectively At least part of the motion information unit of the candidate motion information unit set corresponding to the different pixel samples, wherein the motion information unit includes a motion vector whose prediction direction is a forward motion vector and/or a prediction direction is a backward motion vector;
  • the prediction unit 530 is configured to perform pixel value prediction on the image block x by using the non-translating motion model and the combined motion information unit set i.
  • the second determining unit 520 is specifically configured to: determine, from among the N candidate motion information unit sets, a combined motion information unit set i that includes K1 motion information units; wherein the N candidate combined motion information unit sets Each motion information contained in each candidate merged motion information unit set And a unit that is selected from the candidate motion information unit set corresponding to the different pixel samples in the K1 pixel samples, respectively, and the at least part of the motion information unit that meets the constraint, wherein the N is a positive integer, and the N candidates are merged.
  • the sets of motion information units are different from each other, and each set of candidate motion information units in the set of N candidate motion information units includes K1 motion information units.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition ,
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the second condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction.
  • the third condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same reference frame index.
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a horizontal component threshold.
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets is less than or equal to a vertical component threshold.
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and at least 2 pixel samples in the central pixel sample a1 of the image block x. .
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x including the upper left vertex of the image block x;
  • the lower left pixel sample of the image block x a lower left vertex of the image block x or a pixel block in the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex or a location of the image block x a pixel block in the image block x containing the upper right vertex of the image block x;
  • the central sample a1 of the image block x is a central pixel of the image block x or a pixel block of the image block x that includes a central pixel of the image block x.
  • the candidate motion information unit set corresponding to the upper left pixel sample of the image block x includes motion information units of x1 pixel samples, where the x1 pixel samples include At least one pixel sample adjacent to the upper left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper left pixel sample time domain of the image block x, the x1 being a positive integer.
  • the x1 pixel samples include an upper left pixel of the image block x and a video frame adjacent to a time domain of a video frame to which the image block x belongs. a pixel sample having the same sample position, a spatially adjacent pixel sample on the left side of the image block x, a spatially adjacent pixel sample on the upper left of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x at least one.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block x includes motion information units of x2 pixel samples, where the x2 pixel samples include At least one pixel sample adjacent to the upper right pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper right pixel sample time domain of the image block x, the x2 being a positive integer.
  • the x2 pixel samples include an upper right pixel of the image block x and a video frame adjacent to a time domain of a video frame to which the image block x belongs. a pixel sample having the same sample position, a spatial adjacent pixel sample on the right side of the image block x, a spatially adjacent pixel sample on the upper right of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x at least one.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block x includes motion information units of x3 pixel samples, where the x3 pixel samples include At least one pixel sample adjacent to a lower left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to a lower left pixel sample time domain of the image block x, the x3 being a positive integer.
  • the x3 pixel samples are included in the figure a pixel sample having the same position as a lower left pixel sample of the image block x, a spatial adjacent pixel sample to the left of the image block x, and the image, among video frames adjacent to the time domain of the video frame to which the block x belongs. At least one of a spatially adjacent pixel sample of the lower left of the block x and a spatially adjacent pixel sample of the lower side of the image block x.
  • the set of candidate motion information units corresponding to the central pixel sample a1 of the image block x includes motion information units of x5 pixel samples, where the x5 pixel samples are included.
  • One of the pixel samples is the pixel sample a2
  • the position of the central pixel sample a1 in the video frame to which the image block x belongs is the same as the position of the pixel sample a2 in the adjacent video frame of the video frame to which the image block x belongs, and the x5 is A positive integer.
  • the prediction unit 530 is specifically configured to: if the prediction direction in the merged motion information unit set i is different from the reference frame index corresponding to the motion vector in the first prediction direction, perform the combined motion information unit set i
  • the scaling process is such that the motion vector in the first motion direction in the merged motion information unit set i is scaled to the same reference frame, and the non-motion motion model and the merged motion information unit set i after the scaling process are utilized Performing pixel value prediction on the image block x, wherein the first prediction direction is forward or backward;
  • the prediction unit 530 is specifically configured to: when the prediction direction in the merged motion information unit set i is a reference frame index corresponding to the forward motion vector, and the prediction direction in the merged motion information unit set i is If the reference frame index corresponding to the motion vector is different, the merged motion information unit set i is subjected to scaling processing such that the forward motion vector in the merged motion information unit set i is scaled to a forward motion vector The same reference frame and causing the backward motion vector in the merged motion information unit set i to be scaled to the same reference frame, using the non-translational motion model and the merged motion information unit set i
  • the image block x is described for pixel value prediction.
  • the prediction unit 530 is specifically configured to calculate a motion vector of each pixel in the image block x by using a non-translating motion model and the combined motion information unit set i, and using the calculated graph A motion vector of each pixel in the block x determines a predicted pixel value of each pixel in the image block x,
  • the prediction unit 530 is specifically configured to calculate a motion vector of each pixel block in the image block x by using a non-translating motion model and the combined motion information unit set i, and using the calculated image block x
  • the motion vector of each pixel block determines the predicted pixel value of each pixel of each pixel block in the image block x.
  • the first determining unit 510 is further configured to: determine K2 pixel samples in the image block y, and determine a candidate motion information unit set corresponding to each of the K2 pixel samples; wherein the K2 is An integer greater than 1, the image block y is adjacent to the image block x spatial domain, and the candidate motion information unit set corresponding to each of the K2 pixel samples includes at least one motion information unit of the candidate;
  • the second determining unit 520 is further configured to determine a combined motion information unit set j that includes K2 motion information units, where the candidate motion information unit set corresponding to the pixel samples z1 in the K2 pixel samples includes a motion information unit a2, which is obtained based on a motion information unit of the pixel sample z2, where the pixel sample z2 is a pixel sample in the image block x that is less than a threshold value from the pixel sample z1, or The pixel sample z2 is a pixel sample in the image block x that is closest to the pixel sample z1; the K2 motion information units in the merged motion information unit set j are respectively selected from the K2 pixel samples. At least part of the motion information unit of the candidate motion information unit set corresponding to each pixel sample in the constraint condition;
  • the prediction unit 530 is further configured to perform pixel value prediction on the image block y by using the non-translating motion model and the combined motion information unit set j.
  • the non-translational motion model may be any one of the following models: an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, and a shear motion. Models, scaled motion models, and bilinear motion models.
  • the image prediction device 500 can be applied to a video encoding device or the image prediction device 500 can be applied to a video decoding device.
  • the second determining unit 520 may be specifically configured to determine, according to the distortion or rate distortion cost, the merged motion information unit set including the K1 motion information units from among the N candidate combined motion information unit sets. .
  • the prediction unit 530 when the image prediction apparatus is applied to the video encoding apparatus, the prediction unit 530 is further configured to write the identifier of the merged motion information unit set i Into the video stream.
  • the second determining unit is specifically configured to: based on a merge motion obtained from a video code stream.
  • the image prediction device 500 can be any device that needs to output and play video, such as a notebook computer, a tablet computer, a personal computer, a mobile phone, and the like.
  • the image prediction apparatus 500 of the present embodiment performs pixel value prediction on the image block x by using the non-motion motion model and the combined motion information unit set i, and merges each motion information in the motion information unit set i.
  • the unit is respectively selected from at least part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples, and the selected range of the combined motion information unit set i is relatively small, and the traditional technology is adopted.
  • the mechanism of filtering out one motion information unit of K1 pixel samples by a large number of calculations in all possible candidate motion information element sets of K1 pixel samples is beneficial to improve coding efficiency and also to reduce non-translational motion based on
  • the model performs the computational complexity of image prediction, which in turn makes it possible to introduce a non-translational motion model into the video coding standard. And because the introduction of the non-translational motion model, it is more conducive to more accurately describe the motion of the object, so there is Conducive to improving the accuracy of prediction.
  • FIG. 6 is a schematic diagram of an image prediction apparatus 600 according to an embodiment of the present invention.
  • the image prediction apparatus 600 may include at least one bus 601, at least one processor 602 connected to the bus 601, and at least one memory connected to the bus 601. 603.
  • the processor 602 calls, by using the bus 601, the code stored in the memory 603 to determine K1 pixel samples in the image block x, and determines candidate motion information corresponding to each of the K1 pixel samples.
  • a set of candidate motion information units corresponding to each of the pixel samples includes at least one motion information unit of the candidate, the K1 being an integer greater than or equal to 2; determining a combined motion information unit set including K1 motion information units I, wherein each of the motion information unit i is selected from at least a part of the motion information unit of the candidate motion information unit group corresponding to the different one of the K1 pixel samples, where
  • the motion information unit includes a motion vector in which the prediction direction is a forward motion vector and/or a prediction direction is a backward motion vector; and the pixel value is performed on the image block x by using the non-transition motion motion model and the combined motion information unit set i. prediction.
  • the processor 602 is configured to determine, from among a set of N candidate combined motion information units, a combined motion information unit set i that includes K1 motion information units, where each of the N candidate combined motion information unit sets Each of the motion information units included in the candidate combined motion information unit set is selected from at least a part of the motion information unit of the candidate motion information unit set corresponding to the different pixel samples in the K1 pixel samples, where The N is a positive integer, the set of N candidate combined motion information units are different from each other, and each set of candidate combined motion information units in the N candidate combined motion information unit sets includes K1 motion information units.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition ,
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the second condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction.
  • the third condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same reference frame index.
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a horizontal component threshold.
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets is less than or equal to a vertical component threshold.
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and at least 2 pixel samples in the central pixel sample a1 of the image block x. .
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x including the upper left vertex of the image block x;
  • the lower left pixel sample of the image block x a lower left vertex of the image block x or a pixel block in the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex or a location of the image block x a pixel block in the image block x that includes the upper right vertex of the image block x;
  • a central pixel sample a1 of the image block x is a central pixel point of the image block x or the image block x includes the A block of pixels at the center pixel of image block x.
  • the candidate motion information unit set corresponding to the upper left pixel sample of the image block x includes motion information units of x1 pixel samples, where the x1 pixel samples include At least one pixel sample adjacent to the upper left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper left pixel sample time domain of the image block x, the x1 being a positive integer.
  • the x1 pixel samples include an upper left pixel of the image block x and a video frame adjacent to a time domain of a video frame to which the image block x belongs. a pixel sample having the same sample position, a spatial adjacent pixel sample on the left side of the image block x, the image At least one of a spatially adjacent pixel sample of the upper left of the block x and a spatially adjacent pixel sample of the upper side of the image block x.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block x includes motion information units of x2 pixel samples, where the x2 pixel samples include At least one pixel sample adjacent to the upper right pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper right pixel sample time domain of the image block x, the x2 being a positive integer.
  • the x2 pixel samples include an upper right pixel of the image block x and a video frame adjacent to a time domain of a video frame to which the image block x belongs. a pixel sample having the same sample position, a spatial adjacent pixel sample on the right side of the image block x, a spatially adjacent pixel sample on the upper right of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x at least one.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block x includes motion information units of x3 pixel samples, where the x3 pixel samples include At least one pixel sample adjacent to a lower left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to a lower left pixel sample time domain of the image block x, the x3 being a positive integer.
  • the x3 pixel samples include a lower left pixel sample of the image block x and a video frame adjacent to a video frame time domain to which the image block x belongs. At least one of a pixel sample of the same position, a spatial neighboring pixel sample of the left side of the image block x, a spatially adjacent pixel sample of the lower left of the image block x, and a spatially adjacent pixel sample of the lower side of the image block x One.
  • the set of candidate motion information units corresponding to the central pixel sample a1 of the image block x includes motion information units of x5 pixel samples, where the x5 pixel samples are included.
  • One of the pixel samples is the pixel sample a2
  • the position of the central pixel sample a1 in the video frame to which the image block x belongs is the same as the position of the pixel sample a2 in the adjacent video frame of the video frame to which the image block x belongs, and the x5 is A positive integer.
  • the processor 602 is configured to: when the prediction direction in the merged motion information unit set i is different, the reference frame index corresponding to the motion vector in the first prediction direction is different, and the merged motion information unit set i is scaled Processing, such that the motion vector in the first motion direction in the merged motion information unit set i is scaled to the same reference frame, using the non-motion motion model and the merged motion information unit set i after performing the scaling process
  • the image block x performs pixel value prediction, wherein the first prediction direction is forward or backward;
  • the processor 602 is configured to: when a prediction direction in the merged motion information unit set i is a reference frame index corresponding to a forward motion vector, and a prediction direction in the merged motion information unit set i is backward If the reference frame index corresponding to the motion vector is different, the merged motion information unit set i is subjected to scaling processing such that the forward motion vector in the merged motion information unit set i is scaled to the same motion vector Referencing the frame and causing the backward direction motion vector in the merged motion information unit set i to be scaled to the same reference frame, using the non-translational motion model and the scaled motion information unit set i Image block x performs pixel value prediction.
  • the processor 602 is configured to calculate a motion vector of each pixel in the image block x by using a non-translational motion model and the combined motion information unit set i, by using the calculated image block x A motion vector of each pixel determines a predicted pixel value of each pixel in the image block x,
  • the processor 602 is configured to calculate a motion vector of each pixel block in the image block x by using a non-translating motion model and the combined motion information unit set i, by using the calculated image block x
  • the motion vector of each pixel block determines a predicted pixel value of each pixel of each pixel block in the image block x.
  • the processor 602 is further configured to: determine K2 pixel samples in the image block y, and determine a candidate motion information unit set corresponding to each of the K2 pixel samples; wherein the K2 is greater than 1.
  • the corresponding candidate motion information unit set includes a motion information unit a2, which is obtained based on a motion information unit of the pixel sample z2, where the pixel sample z2 is less than a distance from the pixel sample z1 in the image block x.
  • the pixel sample z2 is a pixel sample of the image block x that is closest to the pixel sample z1; the K2 motion information units in the merged motion information unit set j, respectively ???selecting at least part of the motion information unit of the candidate motion information unit set corresponding to each of the K2 pixel samples; using the non-translational motion model and the combined motion information unit set
  • the image block y performs pixel value prediction.
  • the non-translational motion model may be any one of the following models: an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, and a shear motion. Models, scaled motion models, and bilinear motion models.
  • the image prediction device 500 can be applied to a video encoding device or the image prediction device 500 can be applied to a video decoding device.
  • the processor 602 may be specifically configured to use N candidate motions according to distortion or rate distortion cost.
  • a combined motion information unit set i including K1 motion information units is determined.
  • the processor 602 may further be configured to identify the combined motion information unit set i Write to the video stream.
  • the processor 602 may be specifically configured to use the combined motion information obtained from the video code stream.
  • the image prediction device 600 can be any device that needs to output and play video, such as a notebook computer, a tablet computer, a personal computer, a mobile phone, and the like.
  • the image prediction apparatus 600 of the present embodiment performs pixel value prediction on the image block x by using the non-motion motion model and the combined motion information unit set i, and merges each motion information in the motion information unit set i.
  • the unit is respectively selected from at least part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples, and the selected range of the combined motion information unit set i is relatively small, and the traditional technology is adopted.
  • the mechanism of filtering out one motion information unit of K1 pixel samples by a large number of calculations in all possible candidate motion information element sets of K1 pixel samples is beneficial to improve coding efficiency and also to reduce non-translational motion based on
  • the model performs the computational complexity of image prediction, which in turn makes it possible to introduce a non-translational motion model into the video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • FIG. 7 is a structural block diagram of an image prediction apparatus 700 according to another embodiment of the present invention.
  • the image prediction apparatus 700 may include at least one processor 701, a memory 705, and at least one communication bus 702. Among them, the communication bus 702 is used to implement connection communication between these components.
  • the image prediction apparatus 700 may optionally include at least one network interface 704 and/or a user interface 703, and the user interface 703 may include a display (eg, a touch screen, an LCD, a Holographic, a CRT, or a Projector). ), click on a device (such as a mouse or trackball touchpad or touch screen, etc.), a camera and/or a pickup device, and the like.
  • a display eg, a touch screen, an LCD, a Holographic, a CRT, or a Projector
  • click on a device such as a mouse or trackball touchpad or touch screen, etc.
  • a camera and/or a pickup device and the like.
  • the memory 705 may include a read only memory and a random access memory, and provides instructions and data to the processor 701. A portion of the memory 705 may also include non-volatile random access memory Reservoir.
  • the memory 705 stores the following elements, executable modules or data structures, or a subset thereof, or their extended set:
  • the operating system 7051 includes various system programs for implementing various basic services and handling hardware-based tasks.
  • the application module 7052 includes various applications for implementing various application services.
  • the processor 701 is configured to determine K1 pixel samples in the image block x by calling a program or instruction stored in the memory 705, and determine that each of the K1 pixel samples corresponds to the pixel sample. a set of candidate motion information units; the set of candidate motion information units corresponding to each of the pixel samples includes at least one motion information unit of the candidate, the K1 being an integer greater than or equal to 2; determining to merge the K1 motion information units a motion information unit set i, wherein each of the motion information unit sets i is selected from at least part of motion information in a candidate motion information unit set corresponding to a different one of the K1 pixel samples a unit, wherein the motion information unit includes a motion vector in which a prediction direction is a forward motion vector and/or a prediction direction is a backward motion vector; and the image block is obtained by using a non-motion motion model and the combined motion information unit set i x performs pixel value prediction.
  • the processor 701 is configured to determine, from among a set of N candidate combined motion information units, a combined motion information unit set i that includes K1 motion information units, where each of the N candidate combined motion information unit sets Each of the motion information units included in the candidate combined motion information unit set is selected from at least a part of the motion information unit of the candidate motion information unit set corresponding to the different pixel samples in the K1 pixel samples, where The N is a positive integer, the set of N candidate combined motion information units are different from each other, and each of the N candidate combined motion information unit sets The candidate merged motion information unit set includes K1 motion information units.
  • the set of N candidate combined motion information units satisfies at least one of a first condition, a second condition, a third condition, a fourth condition, and a fifth condition ,
  • the first condition includes that the motion mode of the image block x indicated by the motion information unit in any one of the N candidate motion information unit sets is a non-translation motion.
  • the second condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same prediction direction.
  • the third condition includes that at least two motion information units in any one of the candidate motion information unit sets in the N candidate combined motion information unit sets have the same reference frame index.
  • the fourth condition includes that an absolute value of a difference value of horizontal components of two motion information units in any one of the N candidate combined motion information unit sets is less than or equal to a horizontal component threshold.
  • the fifth condition includes that an absolute value of a difference of vertical components of two motion information units in any one of the N candidate motion information unit sets in the N candidate motion information unit sets is less than or equal to a vertical component threshold.
  • the K1 pixel samples include an upper left pixel sample, an upper right pixel sample, a lower left pixel sample, and at least 2 pixel samples in the central pixel sample a1 of the image block x. .
  • the upper left pixel sample of the image block x is the upper left vertex of the image block x or the pixel block of the image block x including the upper left vertex of the image block x;
  • the lower left pixel sample of the image block x a lower left vertex of the image block x or a pixel block in the image block x including a lower left vertex of the image block x;
  • an upper right pixel sample of the image block x is an upper right vertex or a location of the image block x a pixel block in the image block x that includes the upper right vertex of the image block x;
  • a central pixel sample a1 of the image block x is a central pixel point of the image block x or the image block x includes the A block of pixels at the center pixel of image block x.
  • the upper left pixel sample of the image block x The corresponding set of candidate motion information units includes motion information units of x1 pixel samples, wherein the x1 pixel samples include at least one pixel sample adjacent to an upper left pixel sample spatial domain of the image block x and/or at least one The pixel sample adjacent to the time domain of the upper left pixel sample of the image block x, the x1 being a positive integer.
  • the x1 pixel samples include an upper left pixel of the image block x and a video frame adjacent to a time domain of a video frame to which the image block x belongs. a pixel sample having the same sample position, a spatially adjacent pixel sample on the left side of the image block x, a spatially adjacent pixel sample on the upper left of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x at least one.
  • the candidate motion information unit set corresponding to the upper right pixel sample of the image block x includes motion information units of x2 pixel samples, where the x2 pixel samples include At least one pixel sample adjacent to the upper right pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to the upper right pixel sample time domain of the image block x, the x2 being a positive integer.
  • the x2 pixel samples include an upper right pixel of the image block x and a video frame adjacent to a time domain of a video frame to which the image block x belongs. a pixel sample having the same sample position, a spatial adjacent pixel sample on the right side of the image block x, a spatially adjacent pixel sample on the upper right of the image block x, and a spatially adjacent pixel sample on the upper side of the image block x at least one.
  • the candidate motion information unit set corresponding to the lower left pixel sample of the image block x includes motion information units of x3 pixel samples, where the x3 pixel samples include At least one pixel sample adjacent to a lower left pixel sample spatial domain of the image block x and/or at least one pixel sample adjacent to a lower left pixel sample time domain of the image block x, the x3 being a positive integer.
  • the x3 pixel samples include a lower left pixel sample of the image block x and a video frame adjacent to a video frame time domain to which the image block x belongs. a pixel sample of the same position, a spatial neighboring pixel sample of the left side of the image block x, a spatially adjacent pixel sample of the lower left of the image block x, and a spatially adjacent pixel sample of the lower side of the image block x missing one.
  • the set of candidate motion information units corresponding to the central pixel sample a1 of the image block x includes motion information units of x5 pixel samples, where the x5 pixel samples are included.
  • One of the pixel samples is the pixel sample a2
  • the position of the central pixel sample a1 in the video frame to which the image block x belongs is the same as the position of the pixel sample a2 in the adjacent video frame of the video frame to which the image block x belongs, and the x5 is A positive integer.
  • the processor 701 is configured to: when the prediction direction in the merged motion information unit set i is different, the reference frame index corresponding to the motion vector in the first prediction direction is different, and the merged motion information unit set i is scaled Processing, such that the motion vector in the first motion direction in the merged motion information unit set i is scaled to the same reference frame, using the non-motion motion model and the merged motion information unit set i after performing the scaling process
  • the image block x performs pixel value prediction, wherein the first prediction direction is forward or backward;
  • the processor 701 may be configured to: when the prediction direction in the merged motion information unit set i is a reference frame index corresponding to a forward motion vector, and the prediction direction in the merged motion information unit set i is If the reference frame index corresponding to the backward motion vector is different, the merged motion information unit set i is subjected to scaling processing such that the forward motion vector in the merged motion information unit set i is scaled by the forward motion vector Moving to the same reference frame and causing the backward direction motion vector in the merged motion information unit set i to be scaled to the same reference frame, using the non-translational motion model and the merged motion information unit set i after scaling processing
  • the image block x performs pixel value prediction.
  • the processor 701 is configured to calculate a motion vector of each pixel in the image block x by using a non-translating motion model and the combined motion information unit set i, by using the calculated image block x A motion vector of each pixel determines a predicted pixel value of each pixel in the image block x,
  • the processor 701 may be configured to calculate a motion vector of each pixel block in the image block x by using a non-translating motion model and the combined motion information unit set i, using the calculated image block x.
  • the motion vector of each pixel block in the image determines the pixel points of each pixel block in the image block x The predicted pixel value.
  • the processor 701 is further configured to: determine K2 pixel samples in the image block y, and determine a candidate motion information unit set corresponding to each of the K2 pixel samples; wherein the K2 is greater than 1.
  • An integer the image block y is adjacent to the image block x spatial domain, and the candidate motion information unit set corresponding to each of the K2 pixel samples includes at least one motion information unit of the candidate; a set of merged motion information units j of the motion information units; wherein the set of candidate motion information units corresponding to the pixel samples z1 of the K2 pixel samples includes a motion information unit a2, which is based on the pixel samples z2 Obtaining, by the motion information unit, the pixel sample z2 is a pixel sample of the image block x whose distance from the pixel sample z1 is less than a threshold, or the pixel sample z2 is the image block x and the pixel sample Z1 is the nearest pixel sample; the K2 motion information units in the merged motion information unit set
  • the non-translational motion model may be any one of the following models: an affine motion model, a parabolic motion model, a rotational motion model, a perspective motion model, and a shear motion. Models, scaled motion models, and bilinear motion models.
  • the image prediction device 500 can be applied to a video encoding device or the image prediction device 500 can be applied to a video decoding device.
  • the processor 701 may be specifically configured to use N candidate motions according to distortion or rate distortion cost.
  • a combined motion information unit set i including K1 motion information units is determined.
  • the processor 701 may further be configured to use the identifier of the merged motion information unit set i Write to the video stream.
  • the processor 701 may be specifically configured to determine, according to the identifier of the merged motion information unit set i obtained from the video code stream, that the K1 motions are included from the N candidate combined motion information unit sets.
  • the merged motion information unit set i of the information unit may be specifically configured to determine, according to the identifier of the merged motion information unit set i obtained from the video code stream, that the K1 motions are included from the N candidate combined motion information unit sets.
  • the image prediction device 700 can be any device that needs to output and play video, such as a notebook computer, a tablet computer, a personal computer, a mobile phone, and the like.
  • the image prediction apparatus 700 of the present embodiment performs pixel value prediction on the image block x by using the non-motion motion model and the combined motion information unit set i, and merges each motion information in the motion information unit set i.
  • the unit is respectively selected from at least part of the motion information unit of the candidate motion information unit group corresponding to the different pixel samples in the K1 pixel samples, and the selected range of the combined motion information unit set i is relatively small, and the traditional technology is adopted.
  • the mechanism of filtering out one motion information unit of K1 pixel samples by a large number of calculations in all possible candidate motion information element sets of K1 pixel samples is beneficial to improve coding efficiency and also to reduce non-translational motion based on
  • the model performs the computational complexity of image prediction, which in turn makes it possible to introduce a non-translational motion model into the video coding standard. And because the non-translational motion model is introduced, it is beneficial to describe the motion of the object more accurately, so it is beneficial to improve the prediction accuracy.
  • the disclosed apparatus may be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the above units is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or integrated. Go to another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical or otherwise.
  • the units described above as separate components may or may not be physically separated.
  • the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the above-described integrated unit if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-described methods of various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Image Analysis (AREA)
PCT/CN2015/077295 2014-10-27 2015-04-23 图像预测方法及相关装置 Ceased WO2016065873A1 (zh)

Priority Applications (13)

Application Number Priority Date Filing Date Title
JP2017517269A JP6490203B2 (ja) 2014-10-27 2015-04-23 画像予測方法および関連装置
KR1020217012948A KR102334151B1 (ko) 2014-10-27 2015-04-23 이미지 예측 방법 및 관련 장치
KR1020177007312A KR102013771B1 (ko) 2014-10-27 2015-04-23 이미지 예측 방법 및 관련 장치
BR112017007399-4A BR112017007399B1 (pt) 2014-10-27 2015-04-23 Método de predição de imagem e aparelho relacionado
EP15855692.8A EP3177015B1 (en) 2014-10-27 2015-04-23 Image prediction method and related device
EP24154875.9A EP4387229A3 (en) 2014-10-27 2015-04-23 Picture prediction method and related apparatus
KR1020197024188A KR102070982B1 (ko) 2014-10-27 2015-04-23 이미지 예측 방법 및 관련 장치
KR1020207002096A KR102248254B1 (ko) 2014-10-27 2015-04-23 이미지 예측 방법 및 관련 장치
US15/463,850 US10440380B2 (en) 2014-10-27 2017-03-20 Picture prediction method and related apparatus
US16/431,298 US10623763B2 (en) 2014-10-27 2019-06-04 Picture prediction method and related apparatus
US16/845,161 US11172217B2 (en) 2014-10-27 2020-04-10 Picture prediction method and related apparatus
US17/511,269 US11968386B2 (en) 2014-10-27 2021-10-26 Picture prediction method and related apparatus
US18/628,367 US12382081B2 (en) 2014-10-27 2024-04-05 Picture prediction method and related apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410584175.1A CN104363451B (zh) 2014-10-27 2014-10-27 图像预测方法及相关装置
CN201410584175.1 2014-10-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/463,850 Continuation US10440380B2 (en) 2014-10-27 2017-03-20 Picture prediction method and related apparatus

Publications (1)

Publication Number Publication Date
WO2016065873A1 true WO2016065873A1 (zh) 2016-05-06

Family

ID=52530673

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/077295 Ceased WO2016065873A1 (zh) 2014-10-27 2015-04-23 图像预测方法及相关装置

Country Status (6)

Country Link
US (5) US10440380B2 (enExample)
EP (2) EP4387229A3 (enExample)
JP (2) JP6490203B2 (enExample)
KR (4) KR102248254B1 (enExample)
CN (1) CN104363451B (enExample)
WO (1) WO2016065873A1 (enExample)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109792533A (zh) * 2016-10-05 2019-05-21 高通股份有限公司 在视频译码中用于仿射运动模型的运动向量预测
EP3457694A4 (en) * 2016-05-24 2019-05-22 Huawei Technologies Co., Ltd. IMAGE FORECASTING METHOD AND ASSOCIATED DEVICE
JP2021502031A (ja) * 2017-11-07 2021-01-21 華為技術有限公司Ｈｕａｗｅｉ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｃｏ．，Ｌｔｄ． ビデオ符号化のためのインター予測機器及び方法の補間フィルタ

Families Citing this family (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104363451B (zh) 2014-10-27 2019-01-25 华为技术有限公司 图像预测方法及相关装置
RU2671307C1 (ru) 2015-03-10 2018-10-30 Хуавэй Текнолоджиз Ко., Лтд. Способ предсказания изображений и связанное устройство
CN106303543B (zh) 2015-05-15 2018-10-30 华为技术有限公司 视频图像编码和解码的方法、编码设备和解码设备
CN106331722B (zh) 2015-07-03 2019-04-26 华为技术有限公司 图像预测方法和相关设备
CN105163116B (zh) 2015-08-29 2018-07-31 华为技术有限公司 图像预测的方法及设备
CN108965871B (zh) 2015-09-29 2023-11-10 华为技术有限公司 图像预测的方法及装置
KR20180059482A (ko) * 2015-10-22 2018-06-04 엘지전자 주식회사 비디오 코딩 시스템에서 인트라 예측 방법 및 장치
CN105681807B (zh) * 2016-01-06 2018-11-02 福州瑞芯微电子股份有限公司 一种基于h264协议的分像素运动矢量计算方法和装置
CN108432250A (zh) * 2016-01-07 2018-08-21 联发科技股份有限公司 用于视频编解码的仿射帧间预测的方法及装置
WO2017146526A1 (ko) * 2016-02-25 2017-08-31 주식회사 케이티 비디오 신호 처리 방법 및 장치
WO2017156705A1 (en) * 2016-03-15 2017-09-21 Mediatek Inc. Affine prediction for video coding
EP4614977A3 (en) 2016-03-15 2025-12-03 HFI Innovation Inc. Method and apparatus of video coding with affine motion compensation
US10560712B2 (en) 2016-05-16 2020-02-11 Qualcomm Incorporated Affine motion prediction for video coding
CN108271023B (zh) * 2017-01-04 2021-11-19 华为技术有限公司 图像预测方法和相关设备
CN107318024A (zh) * 2017-06-27 2017-11-03 北京奇艺世纪科技有限公司 基于运动场的视频编码方法及装置
US11877001B2 (en) 2017-10-10 2024-01-16 Qualcomm Incorporated Affine prediction in video coding
WO2019117659A1 (ko) * 2017-12-14 2019-06-20 엘지전자 주식회사 움직임 벡터 도출을 기반으로 하는 영상 코딩 방법 및 그 장치
US20190208211A1 (en) * 2018-01-04 2019-07-04 Qualcomm Incorporated Generated affine motion vectors
US11245919B2 (en) 2018-03-16 2022-02-08 Sony Corporation Image processing device and image processing method
WO2019203504A1 (ko) * 2018-04-15 2019-10-24 엘지전자 주식회사 비디오 처리 시스템에서 움직임 정보에 대한 저장 공간을 감소시키는 방법 및 장치
CN116800981A (zh) * 2018-05-23 2023-09-22 寰发股份有限公司 使用双向编码单元加权的视频编码方法和装置
WO2019244719A1 (en) * 2018-06-18 2019-12-26 Sharp Kabushiki Kaisha Systems and methods for performing affine motion compensation prediction for coding of video data
CN110620923B (zh) 2018-06-19 2022-07-12 北京字节跳动网络技术有限公司 广义的运动矢量差分辨率
EP3817382A4 (en) * 2018-07-11 2021-07-28 Huawei Technologies Co., Ltd. VIDEO ENCODER, VIDEO DECODER AND RELEVANT PROCEDURE
WO2020013569A1 (ko) 2018-07-13 2020-01-16 엘지전자 주식회사 영상 코딩 시스템에서 어파인 움직임 예측에 기반한 영상 디코딩 방법 및 장치
WO2020032609A1 (ko) * 2018-08-09 2020-02-13 엘지전자 주식회사 영상 코딩 시스템에서 어파인 머지 후보 리스트를 사용하는 어파인 움직임 예측에 기반한 영상 디코딩 방법 및 장치
WO2020038232A1 (zh) * 2018-08-21 2020-02-27 华为技术有限公司 一种图像块的运动信息的预测方法及装置
KR102635047B1 (ko) 2018-09-19 2024-02-07 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 적응적 움직임 벡터 해상도를 가지는 어파인 모드에 대한 구문 재사용
CN114866777B (zh) * 2018-09-20 2024-11-22 杭州海康威视数字技术股份有限公司 一种解码、编码方法及其设备
CN112740663B (zh) * 2018-09-24 2022-06-14 华为技术有限公司 图像预测方法、装置以及相应的编码器和解码器
SG11202104022XA (en) * 2018-11-02 2021-05-28 Beijing Bytedance Network Technology Co Ltd Table maintenance for hmvp candidate storage
CN113228635B (zh) 2018-11-13 2024-01-05 北京字节跳动网络技术有限公司 用于帧内块复制的运动候选列表构建方法
US11024041B2 (en) * 2018-12-10 2021-06-01 Intel Corporation Depth and motion estimations in machine learning environments
CN113366851B (zh) 2019-01-31 2025-10-21 北京字节跳动网络技术有限公司 对称运动矢量差编解码模式的快速算法
WO2020156516A1 (en) 2019-01-31 2020-08-06 Beijing Bytedance Network Technology Co., Ltd. Context for coding affine mode adaptive motion vector resolution
CN111526362B (zh) * 2019-02-01 2023-12-29 华为技术有限公司 帧间预测方法和装置
CN110809161B (zh) * 2019-03-11 2020-12-29 杭州海康威视数字技术股份有限公司 运动信息候选者列表构建方法及装置
CN113812165B (zh) 2019-05-09 2023-05-23 北京字节跳动网络技术有限公司 对hmvp表的改进
WO2021056205A1 (zh) * 2019-09-24 2021-04-01 北京大学 一种视频处理方法、设备及存储介质
US20230134603A1 (en) * 2020-05-01 2023-05-04 Given Imaging Ltd. Systems and methods for selecting images of event indicators
US20220319019A1 (en) * 2021-03-31 2022-10-06 Nvidia Corporation Techniques to determine optical flow
US11936877B2 (en) * 2021-04-12 2024-03-19 Qualcomm Incorporated Template matching based affine prediction for video coding
WO2025150796A1 (ko) * 2024-01-08 2025-07-17 주식회사 엘엑스 세미콘 인터 예측 기반 영상 디코딩 방법 및 그 장치

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060125956A1 (en) * 2004-11-17 2006-06-15 Samsung Electronics Co., Ltd. Deinterlacing method and device in use of field variable partition type
CN101350928A (zh) * 2008-07-29 2009-01-21 北京中星微电子有限公司 一种运动估计方法及装置
CN102883160A (zh) * 2009-06-26 2013-01-16 华为技术有限公司 视频图像运动信息获取方法、装置及设备、模板构造方法
CN104363451A (zh) * 2014-10-27 2015-02-18 华为技术有限公司 图像预测方法及相关装置

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW257924B (en) 1995-03-18 1995-09-21 Daewoo Electronics Co Ltd Method and apparatus for encoding a video signal using feature point based motion estimation
JP2798120B2 (ja) * 1995-08-04 1998-09-17 日本電気株式会社 動き補償フレーム間予測方法及び動き補償フレーム間予測装置
US5959673A (en) 1995-10-05 1999-09-28 Microsoft Corporation Transform coding of dense motion vector fields for frame and object based video coding applications
JPH104555A (ja) * 1996-06-17 1998-01-06 Oki Electric Ind Co Ltd 動きベクトル検出装置
JP2000350216A (ja) * 1999-06-09 2000-12-15 Matsushita Electric Ind Co Ltd 動き係数探索装置、及び動き係数探索方法
US6735249B1 (en) * 1999-08-11 2004-05-11 Nokia Corporation Apparatus, and associated method, for forming a compressed motion vector field utilizing predictive motion coding
JP5401071B2 (ja) * 2008-10-09 2014-01-29 株式会社Ｎｔｔドコモ 動画像符号化装置、動画像復号装置、動画像符号化方法、動画像復号方法、動画像符号化プログラム、動画像復号プログラム、動画像処理システムおよび動画像処理方法
KR101279573B1 (ko) * 2008-10-31 2013-06-27 에스케이텔레콤 주식회사 움직임 벡터 부호화 방법 및 장치와 그를 이용한 영상 부호화/복호화 방법 및 장치
US8406569B2 (en) * 2009-01-19 2013-03-26 Sharp Laboratories Of America, Inc. Methods and systems for enhanced dynamic range images and video from multiple exposures
US9131239B2 (en) * 2011-06-20 2015-09-08 Qualcomm Incorporated Unified merge mode and adaptive motion vector prediction mode candidates selection
JP5937594B2 (ja) * 2011-06-30 2016-06-22 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ 画像復号方法及び画像復号装置
CN103891290B (zh) * 2011-08-19 2018-02-02 瑞典爱立信有限公司 运动矢量处理
CN103024378B (zh) * 2012-12-06 2016-04-13 浙江大学 一种视频编解码中运动信息导出方法及装置
CN105122803B (zh) * 2013-04-02 2019-01-08 Vid拓展公司 针对可伸缩视频编码的增强型时间运动向量预测

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060125956A1 (en) * 2004-11-17 2006-06-15 Samsung Electronics Co., Ltd. Deinterlacing method and device in use of field variable partition type
CN101350928A (zh) * 2008-07-29 2009-01-21 北京中星微电子有限公司 一种运动估计方法及装置
CN102883160A (zh) * 2009-06-26 2013-01-16 华为技术有限公司 视频图像运动信息获取方法、装置及设备、模板构造方法
CN104363451A (zh) * 2014-10-27 2015-02-18 华为技术有限公司 图像预测方法及相关装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3177015A4 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3457694A4 (en) * 2016-05-24 2019-05-22 Huawei Technologies Co., Ltd. IMAGE FORECASTING METHOD AND ASSOCIATED DEVICE
CN109792533A (zh) * 2016-10-05 2019-05-21 高通股份有限公司 在视频译码中用于仿射运动模型的运动向量预测
CN109792533B (zh) * 2016-10-05 2023-08-15 高通股份有限公司 一种解码和编码视频数据的方法及装置
JP2021502031A (ja) * 2017-11-07 2021-01-21 華為技術有限公司Ｈｕａｗｅｉ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｃｏ．，Ｌｔｄ． ビデオ符号化のためのインター予測機器及び方法の補間フィルタ
US11297324B2 (en) 2017-11-07 2022-04-05 Huawei Technologies Co., Ltd. Interpolation filter for an inter prediction apparatus and method for video coding
US11375199B2 (en) 2017-11-07 2022-06-28 Huawei Technologies Co., Ltd. Interpolation filter for an inter prediction apparatus and method for video coding

Also Published As

Publication number Publication date
KR20170045264A (ko) 2017-04-26
US10623763B2 (en) 2020-04-14
JP2019041418A (ja) 2019-03-14
EP3177015B1 (en) 2024-04-03
BR112017007399A2 (pt) 2018-06-19
KR20200011568A (ko) 2020-02-03
KR102013771B1 (ko) 2019-08-23
US20220053208A1 (en) 2022-02-17
CN104363451B (zh) 2019-01-25
JP6490203B2 (ja) 2019-03-27
US20170195685A1 (en) 2017-07-06
KR20190100435A (ko) 2019-08-28
KR102334151B1 (ko) 2021-12-02
US11968386B2 (en) 2024-04-23
JP7313816B2 (ja) 2023-07-25
EP3177015A4 (en) 2017-08-09
KR102070982B1 (ko) 2020-01-29
JP2017536002A (ja) 2017-11-30
US11172217B2 (en) 2021-11-09
US12382081B2 (en) 2025-08-05
EP4387229A2 (en) 2024-06-19
US20200244983A1 (en) 2020-07-30
KR20210049989A (ko) 2021-05-06
EP3177015A1 (en) 2017-06-07
CN104363451A (zh) 2015-02-18
US20240276006A1 (en) 2024-08-15
KR102248254B1 (ko) 2021-05-06
US20190289313A1 (en) 2019-09-19
EP4387229A3 (en) 2024-09-04
US10440380B2 (en) 2019-10-08

Similar Documents

Publication Publication Date Title
WO2016065873A1 (zh) 图像预测方法及相关装置
US11178419B2 (en) Picture prediction method and related apparatus
CN107710761B (zh) 图像预测方法和相关设备
WO2016050051A1 (zh) 图像预测方法及相关装置
WO2017005146A1 (zh) 视频编码和解码方法、视频编码和解码装置
WO2017201678A1 (zh) 图像预测方法和相关设备
WO2016065872A1 (zh) 图像预测方法及相关装置
CN104333758A (zh) 深度图的预测方法和检测像素点的方法及相关装置
US20250392742A1 (en) Picture prediction method and related apparatus
HK1243852B (zh) 图像预测方法和相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15855692

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2015855692

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2015855692

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 20177007312

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2017517269

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112017007399

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112017007399

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20170410