CN112055207A - Time domain motion vector prediction method, apparatus and storage medium - Google Patents

Time domain motion vector prediction method, apparatus and storage medium Download PDF

Info

Publication number
CN112055207A
CN112055207A CN202010784588.XA CN202010784588A CN112055207A CN 112055207 A CN112055207 A CN 112055207A CN 202010784588 A CN202010784588 A CN 202010784588A CN 112055207 A CN112055207 A CN 112055207A
Authority
CN
China
Prior art keywords
block
time domain
sub
motion vector
current block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010784588.XA
Other languages
Chinese (zh)
Other versions
CN112055207B (en
Inventor
方诚
江东
曾飞洋
林聚财
殷俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Dahua Technology Co Ltd
Original Assignee
Zhejiang Dahua Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Dahua Technology Co Ltd filed Critical Zhejiang Dahua Technology Co Ltd
Priority to CN202010784588.XA priority Critical patent/CN112055207B/en
Publication of CN112055207A publication Critical patent/CN112055207A/en
Application granted granted Critical
Publication of CN112055207B publication Critical patent/CN112055207B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The application discloses a time domain motion vector prediction method, a device and a storage medium. The time domain motion vector prediction method comprises the following steps: when the method is used for predicting the time domain motion vector based on the subblocks, dividing the current block to obtain a plurality of subblocks with the same size; and acquiring a time domain motion vector corresponding to a pixel block at any position in each sub-block, and determining the time domain motion vector of each sub-block based on the time domain motion vector corresponding to the pixel block. The time domain motion vector corresponding to the pixel block at any position of each subblock is obtained, and the time domain motion vector corresponding to the pixel block is used as the time domain motion vector of each subblock, so that the positions of the subblocks for obtaining the time domain motion vector are more flexible, and the positions of different subblocks for obtaining the time domain motion vector can be different, thereby improving the accuracy of the subblocks for obtaining the time domain motion vector and further improving the accuracy of time domain motion vector prediction.

Description

Time domain motion vector prediction method, apparatus and storage medium
Technical Field
The application belongs to the technical field of video coding and decoding, and particularly relates to a time domain motion vector prediction method, time domain motion vector prediction equipment and a storage medium.
Background
Because the data volume of the video image is large, the main function of video coding is to compress the video pixel data (RGB, YUV, etc.) into a video code stream, thereby reducing the data volume of the video, and achieving the purposes of reducing the network bandwidth and reducing the storage space in the transmission process.
Generally, the luminance and chrominance signal values of the pixels of the temporally adjacent frames are relatively close and have strong correlation. Inter prediction finds a matching block closest to a current block in a reference frame, and records motion information such as a motion vector mv (motion vector) and a reference frame index between the current block and the matching block. And encoding the motion information and transmitting the encoded motion information to a decoding end. At the decoding end, the decoder can find the matching block of the current block as long as the MV of the current block is analyzed through the corresponding syntax element. And copying the pixel value of the matching block to the current block, namely the interframe prediction value of the current block. However, the conventional prediction method for motion information such as motion vector between the current block and the matching block is not accurate enough, and a more accurate prediction method is needed to solve the above problems.
Disclosure of Invention
The application provides a time domain motion vector prediction method, a device and a storage medium, which are used for solving the problem that the motion vector prediction is not accurate enough.
In order to solve the technical problem, the application adopts a technical scheme that: a method of temporal motion vector prediction, the method comprising: when the time domain motion vector prediction is carried out based on the sub-blocks, the current block is divided to obtain a plurality of sub-blocks with the same size; and acquiring a time domain motion vector corresponding to a pixel block at any position in each sub-block, and determining the time domain motion vector of each sub-block based on the time domain motion vector corresponding to the pixel block.
According to an embodiment of the present application, the obtaining a temporal motion vector corresponding to a pixel block at any position in each of the sub-blocks, and determining the temporal motion vector of each of the sub-blocks based on the temporal motion vector corresponding to the pixel block includes: and acquiring a corresponding time domain motion vector by using a pixel block at the position of the upper left corner, the lower left corner, the upper right corner, the lower right corner or the central point of each sub-block to determine the time domain motion vector of each sub-block, wherein the side length of the pixel block is four pixels.
According to an embodiment of the present application, the method further comprises: responding to the current sub-block being unable to obtain a valid time domain motion vector; determining a valid temporal motion vector of the current sub-block based on a temporal motion vector of a block of pixels of another location in the current sub-block; or, obtaining the effective time domain motion vectors of the rest sub-blocks in the current block in a first sequence, and taking the effective time domain motion vectors of the rest sub-blocks in the previous obtaining sequence as the time domain motion vectors of the current sub-block; or, obtaining the effective time domain motion vectors of the first number of the rest sub-blocks in the current block in a second sequence or randomly, and taking the effective time domain motion vectors as the effective time domain motion vectors of the current sub-block after weighted averaging.
According to an embodiment of the present application, the partitioning the current block to obtain a plurality of sub-blocks with the same size includes: and dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and the side length larger than a preset value.
According to an embodiment of the present application, the dividing the current block according to the size of the current block includes: comparing lengths of a first side and a second side of the current block, the first side being adjacent to the second side; in response to that the length ratio of the first side edge to the second side edge is greater than or equal to a first predetermined multiple, evenly dividing the current block in a direction perpendicular to the first side edge, wherein the first predetermined multiple is greater than one; or, in response to that the length ratio of the second side to the first side is greater than or equal to a second predetermined multiple, equally dividing the current block in a direction perpendicular to the second side, where the second predetermined multiple is greater than one; or, in response to the first side being equal to the second side, equally dividing the current block in a cross-partition method.
According to an embodiment of the present application, the evenly dividing the current block according to the size of the current block includes: acquiring the lengths of a first side edge and a second side edge of the current block, wherein the first side edge is adjacent to the second side edge; in response to the length of the first side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the first side; and/or, in response to the length of the second side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the second side.
According to an embodiment of the present application, before the partitioning the current block, the method further includes: acquiring two first time domain motion vectors corresponding to a first group of pixel blocks at the diagonal position of the current block and two second time domain motion vectors corresponding to a second group of pixel blocks at the diagonal position of the current block; judging whether the two first time domain motion vectors are the same and/or whether the two second time domain motion vectors are the same; if so, the current block is not partitioned.
According to an embodiment of the present application, the temporal motion vector prediction based on sub-blocks includes: judging whether the current prediction mode is a skip/direct inter-frame prediction mode, whether the side lengths of a first side and a second side of the current block are both greater than or equal to the preset value, at least one side is greater than or equal to two times of the preset value, and the first side is adjacent to the second side; and if so, performing time domain motion vector prediction based on the sub-blocks.
According to an embodiment of the present application, the method further comprises: and performing motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
In order to solve the above technical problem, the present application adopts another technical solution: a method of temporal motion vector prediction, the method comprising: when time domain motion vector prediction is carried out based on sub-blocks, the current block is divided according to the size of the current block, and a plurality of sub-blocks with the same size and the side length larger than a preset value are obtained; and acquiring a time domain motion vector of each sub-block.
In order to solve the above technical problem, the present application adopts another technical solution: an electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement any one of the temporal motion vector prediction methods described above.
In order to solve the above technical problem, the present application adopts another technical solution: a computer readable storage medium having stored thereon program data which, when executed by a processor, implements any of the temporal motion vector prediction methods described above.
The beneficial effect of this application is: the method comprises the steps of obtaining a time domain motion vector corresponding to a pixel block at any position of each sub-block, and taking the time domain motion vector corresponding to the pixel block as the time domain motion vector of each sub-block. The positions of the sub-blocks for acquiring the time domain motion vectors are more flexible, and the positions of different sub-blocks for acquiring the time domain motion vectors can be different, so that the accuracy of acquiring the time domain motion vectors by the sub-blocks is improved, and the accuracy of predicting the time domain motion vectors is further improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without inventive efforts, wherein:
FIG. 1 is a flowchart illustrating an embodiment of a temporal motion vector prediction method according to the present application;
FIG. 2 is a schematic diagram of a temporal co-located block range corresponding to a current block with different coordinates when a temporal motion vector of the current block is obtained in an embodiment of a temporal motion vector prediction method according to the present application;
FIG. 3 is a schematic diagram of a frame for determining whether neighboring blocks in a spatial domain are available when a temporal motion vector of a current block is obtained according to an embodiment of a temporal motion vector prediction method of the present application;
FIG. 4 is a schematic flow chart illustrating adding a termination condition for starting the sub TMVP technique according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 5 is a block diagram of a method for temporal motion vector prediction according to an embodiment of the present disclosure;
FIG. 6 is a block diagram of a method for temporal motion vector prediction according to an embodiment of the present disclosure;
FIG. 7 is a block diagram illustrating a frame that a sub-block cannot obtain a valid temporal motion vector according to an embodiment of the temporal motion vector prediction method of the present application;
FIG. 8 is a flowchart illustrating a temporal motion vector prediction method according to another embodiment of the present application;
FIG. 9 is a block diagram of an embodiment of a temporal motion vector predictor of the present application;
FIG. 10 is a block diagram of a temporal motion vector predictor according to another embodiment of the present application;
FIG. 11 is a block diagram of an embodiment of an electronic device of the present application;
FIG. 12 is a block diagram of an embodiment of a computer-readable storage medium of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, an embodiment of the present application provides a temporal motion vector prediction method, including the following steps:
s11: when the time domain motion vector prediction is carried out based on the sub-blocks, the current block is divided, and a plurality of sub-blocks with the same size are obtained.
And if the current prediction mode is a skip/direct inter-frame prediction mode and the first side and the second side of the current block are both more than or equal to 16 pixels, the preset enabling condition of the time domain motion vector prediction based on the sub-blocks is met. At this time, temporal motion vector prediction (subTMVP) based on the subblocks may be started), and the current block is divided to obtain a plurality of subblocks with the same size.
In some embodiments, the current block is directly cross-segmented to obtain a plurality of sub-blocks with the same size, and the side length of each sub-block is greater than a predetermined value, specifically, the predetermined value is 4, but in other embodiments, the predetermined value may also be other values.
In still other embodiments, partitioning the current block into sub-blocks of the same size comprises: and dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and the side length larger than a preset value. The segmentation mode is flexibly selected according to the size of the current block, and the prediction accuracy is effectively improved under the condition of controlling the complexity increase amplitude.
The method specifically comprises the following schemes:
A. determining the segmentation mode by using the aspect ratio:
and comparing the lengths of a first side edge and a second side edge of the current block, wherein the first side edge and the second side edge are adjacent. Responding to the length ratio of the first side edge to the second side edge being more than or equal to a first preset multiple r0The first predetermined multiple is greater than one (r)0>1) Equally dividing the current block in a direction perpendicular to the first side into n0(n0Not less than 2) sub-blocks of the same size; or responding to the length ratio of the second side edge to the first side edge being more than or equal to a second preset multiple r1The second predetermined multiple is greater than one r1(r1>1) Equally dividing the current block in a direction perpendicular to the second side edge, the average division being n1(n1Not less than 2) sub-blocks of the same size; averaging the current block in a cross-hatch method in response to the first side being equal to the second sideAnd dividing, wherein the division can be divided for multiple times. The side lengths of the obtained sub-blocks are all required to be larger than or equal to a preset value, otherwise, the division is not carried out.
The following is described in connection with several embodiments:
in one embodiment, if the length of the first side is greater than the length of the second side, the ratio of the first side to the second side is r0The settable rule divides the current block into n along the direction vertical to the first side edge0An individual block; if the length of the second side edge is larger than that of the first side edge, the ratio of the second side edge to the first side edge is r1The current block is divided into n along the direction perpendicular to the second side edge by the setting rule1And (4) sub-blocks.
In one embodiment, the ratio r of the first side edge to the second side edge is determined0Or the ratio r of the second side edge to the first side edge1Dividing the data into different sections from small to large, wherein n in each section0Or n1The values are consistent. For example, when preset 2 ≦ r0When n is less than or equal to 4, n0Are both equal to 2; preset when r is more than or equal to 80When n is less than or equal to 16, n0Are all equal to 8; preset when 16<r0When n is greater than n0Are all equal to 4. n is1The same is true for the setting of (1).
B. Determining a segmentation mode by using the side length of the current block:
and acquiring the lengths of a first side edge and a second side edge of the current block, wherein the first side edge and the second side edge are adjacent. Responsive to the length w of the first side edge0The current block is divided equally in a direction perpendicular to the first side into n, at least twice the predetermined value3Sub-blocks of the same size; and/or, a length h in response to the second side edge0The current block is divided equally in a direction perpendicular to the second side edge by n, at least twice the predetermined value4Sub-blocks of the same size. And after division, ensuring that the side length of each subblock is more than or equal to a preset value, otherwise, not carrying out the division.
The following is described in connection with several embodiments:
in one embodiment, the length w of the first side edge varies0Corresponding to different n3The value of the one or more of,length h of the second side edge0Corresponding to different n4The value is obtained. For example, the length w of the first side edge can be set0The larger, n3The larger; length h of the second side edge0The larger, n4The larger.
In one embodiment, the length w of the first side edge is measured0Or the length h of the second side edge0Dividing the data into different sections from small to large, wherein n in each section3Or n4The values are consistent.
Of course, the partition modes of a and B may also be combined, and the partition mode may be determined by considering the ratio of the first side and the second side of the current block.
In some embodiments, before step S11, it is also necessary to:
and judging whether the current prediction mode and the current block meet preset enabling conditions for performing time domain motion vector prediction based on the subblocks.
And if so, performing time domain motion vector prediction based on the sub-blocks.
Judging whether the current prediction mode and the current block meet preset enabling conditions for performing time domain motion vector prediction based on the subblocks comprises the following steps: judging whether the current prediction mode is a skip/direct inter-frame prediction mode, whether the side lengths of a first side and a second side of the current block are both larger than or equal to a preset value, at least one of the first side and the second side is larger than or equal to two times of the preset value, and the first side is adjacent to the second side. In some embodiments, the first side is the width of the current block, and the second side is the height of the current block, specifically, the predetermined value is 4 pixels, in other embodiments, the predetermined value may also be other values, and only needs to match the minimum side length required by the sub-block, which is not limited herein. Compared with the prior art that both the first side and the second side of the current block need to be more than or equal to 16 pixels, in the method, the size of the current block can be smaller, the side lengths of the sub blocks obtained only by partitioning are more than or equal to 4, and the application is more flexible.
It should be noted that before determining whether the preset enabling condition is met, an MVP (temporal prediction motion vector) candidate list under skip/direct generally needs to be constructed. The list includes several types of MVP modes, which are TMVP (temporal MVP), SMVP (spatial MVP), and HMVP (historical MVP) in order of priority. The temporal motion vector prediction method of the present application is related to TMVP, which is a temporal motion vector (mv) corresponding to the upper left-hand coordinate of the current block.
In the present application, when the TMVP in the candidate list is traversed and a sub-block based temporal motion vector prediction (sub-TMVP) enabling condition is satisfied, the sub-TMVP is enabled, and prediction is performed in a sub-TMVP manner instead of directly using the TMVP after enabling.
The following supplements the acquisition method of the current block TMVP:
step1, obtaining the spatial domain position of the time domain co-location block corresponding to the current block.
The spatial domain position of the current block corresponding to the time domain co-location block is determined firstly, and the spatial domain position of the current co-location block is only related to the current block coordinate and is not related to the current block size.
Let bx, by be the position coordinates of the current block in the whole frame in scu (pixel block of 4 × 4 size), and determine the spatial position of the co-located block by using a mask: mask (-1) ^ 3. And the 'A' represents exclusive OR according to bits, and after exclusive OR is carried out on the-1 and the 3 according to bits, the last two bits of the mask are 0. Next, let the coordinates of the co-block spatial domain position (in scu) be (xpos, ypos):
xpos＝(bx&mask)+2；
ypos＝(by&mask)+2；
where xpos may take the range: bx-1, bx, bx +1, bx + 2; ypos may take the range: by-1, by, by +1, by + 2. The current block at the same coordinate position only corresponds to one time-domain co-location block, the range of the time-domain co-location block corresponding to the current block at different coordinates is shown in fig. 2, the positions of all small dots in fig. 2 are the coordinates of all possible co-location blocks, and each small square represents one scu.
step2, scaling the time domain co-located block MV.
If the current frame is a P frame, the time-domain co-located block is found in the forward co-located reference frame, i.e. index0 frame in the forward reference frame list, and the forward MV of the block is taken as: MV _ col _ f is scaled. Scaling is performed according to the POC distance t1 between the reference frames of index0 in the forward reference frame list of the current frame and the current frame, and the distance t0 between the co-located frame (also index0 in general) to which the forward co-located block belongs and the forward reference frame of the co-located block: scaleMV — MV _ col _ f/t0 t 1. The scaleMV is rounded and the CLIP-to-signed 16bit inner operation is performed to fill the MVP candidate list as TMVP.
If the current frame is a B frame, time domain collocated blocks MV in two directions need to be obtained. The backward co-located reference frame, i.e. the index0 frame in the backward reference frame list, is first found to be the time-domain co-located block, and the forward MV of the block is taken as: the MV _ col _ f performs two scaling operations to obtain MV values in two directions of the TMVP. Wherein, the value of the forward MV is scaled according to the POC distance t1 between the reference frames of index0 in the forward reference frame list of the current frame and the current frame, and the distance t0 between the co-located frame (also generally index0) to which the backward co-located block belongs and the forward reference frame of the co-located block; the value of the backward MV is scaled according to the POC distance t1 between the reference frames of index0 in the backward reference frame list of the current frame and the current frame, and the distance t0 between the co-located frame (also index0 in general) to which the backward co-located block belongs and the forward reference frame of the co-located block. The scaling formula and subsequent operations are the same as those in the case of P frames.
step3, when the MV of the time domain co-located block is not available.
In addition to the above normal situation, there may be a situation where the MV of the time domain co-located block is unavailable, such as a situation where the time domain co-located block is an intra block or IBC block (intra-frame copy block).
The judgment mode is to judge whether the forward reference frame of the time domain co-located block exists. If not, the specific operation is as follows:
if the current frame is a P frame, the TMVP is directly set to 0.
If the current frame is a B frame, the first position of the MVP list is filled with the weighted MVs of the spatial neighboring blocks as the first position of the TMVP candidate list. The rules are as follows:
1) first, it is determined whether neighboring blocks in spatial domain are available, as shown in fig. 3:
firstly, judging whether the solid line blocks a, b and c are available, if the solid line block c is unavailable, taking the dotted line block at the upper left corner as a block c, and judging whether the dotted line block c is available.
2) If only 1 spatial block is available in the three spatial blocks, the MV of the spatial block is scaled and filled into the first position of the MVP candidate list. The scaling is as above.
3) If only 2 spatial blocks are available in the three spatial blocks, the MV of the spatial block is scaled first, and then the following operations are performed:
i. firstly, subtracting the MVs of two spatial domain blocks and taking the absolute value, respectively representing the MVs of three spatial domain positions by mva, mvb and mvc, wherein unavailable MVs are 0,
mva_ext＝abs(mva-mvb)；
mvb_ext＝abs(mvb-mvc)；
mvc_ext＝abs(mvc-mva)；
obtaining the minimum value of the three difference values mva _ ext, mvb _ ext, mvc _ ext, and filling (mva + mvb)/2 at the first position of the MVP candidate list if the minimum value is mva _ ext; if the minimum value is mvb _ ext, filling mvb + mvc)/2; if the minimum value is mvc _ ext, (mvc + mva)/2 is filled.
4) If the MVs of the three spatial blocks are all available, the filling rule after scaling the spatial MVs is as follows:
a) if there is at least one and two other different directions (one positive or two negative or one negative or two positive) under the current component (x or y) of the three MVs:
i. if (mva <0& & mvb >0& & mvc >0) | (mva >0& & mvb <0& & mvc <0), then (mvb + mvc)/2 is filled;
if (mvb <0& & mva >0& & mvc >0) | (mvb >0& & mva <0& & mvc <0), then filling in (mvc + mva)/2);
if (mvc <0& & mva >0& & mvb >0) | (mvc >0& & mva <0& & mvb <0), then filling in (mva + mvb)/2);
b) if the three MVs are all homodromous:
the procedure is the same as in (3).
5) If none is available, 0 is filled.
Since the frame is a B frame, the above operation requires a decision and operation for the forward and backward MVs of the space block.
In some embodiments, before partitioning the current block, a termination condition may be further added for starting the subTMVP technique, as shown in fig. 4, including:
s10: acquiring two first time domain motion vectors corresponding to a first group of pixel blocks at the diagonal position of the current block and two second time domain motion vectors corresponding to a second group of pixel blocks at the diagonal position of the current block;
s101: and judging whether the two first time domain motion vectors are the same and/or whether the two second time domain motion vectors are the same.
S102: if so, the current block is not partitioned.
S103: if not, the current block is segmented.
If not, the process proceeds to step S11.
If so, it means that the current block should be a smaller block, and it is likely that the MVs of the entire block are consistent, so that a sub-block-based temporal motion vector prediction (sub tmvp) technique is not required, which is equivalent to adding a termination condition for starting the sub tmvp technique, and complexity can be reduced while ensuring predictive performance.
The following description is made with reference to a specific embodiment:
as shown in FIG. 5, W0< first side length < W1 and H0< second side length < H1 of the current block, the first side may be the width of the current block and the second side the height of the current block.
I. When the temporal motion vectors of the upper left corner scu1 and the lower right corner scu4 in the current block are the same, the subTMVP is not adopted;
II. When the temporal motion vectors of the upper right corner scu2 and the lower left corner scu3 in the current block are the same, the subTMVP is not adopted;
III, the temporal motion vectors of the upper left corner scu1 and the lower right corner scu4 in the current block are the same, and the temporal motion vectors of the upper right corner scu2 and the lower left corner scu3 are the same without adopting a subTMVP;
IV, temporal motion vectors of the top left corner scu1 and the bottom right corner scu4 in the current block are the same, or temporal motion vectors of the top right corner scu2 and the bottom left corner scu3 are the same without using the sub TMVP.
When the temporal motion vectors of the diagonal scu of the current block are the same, it is likely that the motion vectors of the entire block are consistent, and therefore, the sub tmvp technique is not required.
S12: and acquiring a time domain motion vector corresponding to a pixel block at any position in each sub-block, and determining the time domain motion vector of each sub-block based on the time domain motion vector corresponding to the pixel block.
In an embodiment, the pixel blocks at the upper left corner, lower left corner, upper right corner, lower right corner or the central point in each sub-block are used to obtain the corresponding temporal motion vector to determine the temporal motion vector of each sub-block. And the obtained time domain motion vector of the pixel block needs to be scaled, and the scaled time domain motion vector can be used as the time domain motion vector of the sub-block. In an embodiment of determining the temporal motion vector of each sub-block based on the temporal motion vector corresponding to the pixel block, the step2 in step S11 is the same as the method for scaling the temporal co-located block MV. Of course, in other embodiments, pixel blocks at other positions of each sub-block may be selected, and are not limited herein. Compared with the prior art that only the pixel block at the fixed corner of each subblock can be selected, the method has the advantages that the positions of the subblocks for acquiring the time domain motion vectors are more flexible, and the positions of different subblocks for acquiring the time domain motion vectors can be different, so that the accuracy of acquiring the time domain motion vectors by the subblocks is improved, and the accuracy of predicting the time domain motion vectors is improved.
Specifically, as shown in fig. 6, the current block is divided into four sub-blocks by crosses, and the position where the upper left sub-block 1 selects to obtain the temporal motion vector is the lower right corner scu1 in the current sub-block; the upper right sub-block 2 selects the position for obtaining the temporal motion vector as the lower left corner scu2 in the current sub-block; the lower left sub-block 3 selects the position for obtaining the time domain motion vector as the upper right corner scu3 in the current sub-block; the lower right sub-block 4 selects the position where the temporal motion vector is obtained as the upper left corner scu4 within the current sub-block.
In an embodiment, the current sub-block may not obtain a valid temporal motion vector, e.g., a valid temporal motion vector may not be obtained, or the obtained temporal motion vector may be invalid outside the image. In the prior art, the time domain motion vector of the current block is directly used as the time domain motion vector of the current sub-block, but in the method, after the current sub-block cannot obtain an effective time domain motion vector, the effective time domain motion vector of the current sub-block can be obtained by the following method:
further comprising:
a: a valid temporal motion vector of the current sub-block is determined based on the temporal motion vector of a block of pixels of another position of the current sub-block. Specifically, the remaining pixel blocks in the current sub-block may be searched in a certain order, for example, the pixel blocks at the four corners of the sub-block are searched first, and then the pixel block at the center point is searched, or the pixel blocks are searched directly in a raster scanning order. Until the searching pixel block can obtain the effective time domain motion vector, so as to determine the time domain motion vector of the current sub-block to fill. The searching sequence is filled in the time domain motion vector of the previous adjacent sub-block preferentially, and the searching is not carried out after the filling is finished. And if the effective time domain motion vector is not obtained, taking the time domain motion vector of the current block as the time domain motion vector of the current sub-block.
b: and obtaining the effective time domain motion vectors of the other sub-blocks in the current block in a first sequence, and taking the effective time domain motion vectors of the other sub-blocks in the previous sequence as the time domain motion vectors of the current sub-block. Specifically, the selection of adjacent sub-blocks may specify a first order, such as clockwise, counterclockwise, left-to-right, right-to-left, top-to-bottom, bottom-to-top, etc. And searching similar sub-blocks according to the searching sequence until the effective time domain motion vector is searched and filled as the time domain motion vector of the current sub-block. The searching sequence is filled in the time domain motion vector of the previous adjacent sub-block preferentially, and the searching is not carried out after the filling is finished. And if the effective time domain motion vector is not obtained, taking the time domain motion vector of the current block as the time domain motion vector of the current sub-block.
c: and obtaining the effective time domain motion vectors of the first number of the rest sub-blocks in the current block in a second sequence or randomly, and taking the effective time domain motion vectors as the effective time domain motion vectors of the current sub-block after weighted averaging. The first number n is greater than or equal to 2, and specifically, n (n > ═ 2) sub-blocks in the block are selected in the second order or arbitrarily, and the time domain MV is weighted-averaged to obtain the MV of the current sub-block. And if the effective time domain motion vector is not obtained, taking the time domain motion vector of the current block as the time domain motion vector of the current sub-block.
The following description is made with reference to a specific embodiment:
as shown in fig. 7, in the current 4 sub-blocks, sub-block 4 does not obtain a valid MV, and at this time, scheme b is selected, and a clockwise search is performed with sub-block 3 as a starting point. If the sub-block 3 does not have an effective time domain motion vector, and the sub-block 1 has an effective time domain motion vector, the time domain motion vector of the sub-block 1 is used as the time domain motion vector of the sub-block 4, and the sub-block 2 is not searched any more.
According to the method, the position is more flexible to obtain the time domain motion vector of the sub-block, the time domain motion vector of the similar sub-block can be used for filling when the effective motion vector is not obtained, the correlation is stronger, and the obtained time domain motion vector of the sub-block is more accurate.
S13: and respectively carrying out motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
In an embodiment, motion compensation and distortion cost calculation are performed on the temporal motion vectors of all sub-blocks of the current block, instead of performing motion compensation and distortion cost calculation using the original TMVP mode, so that a more accurate calculation result can be obtained.
As shown in fig. 8, another embodiment of the present method provides a temporal motion vector prediction method, including the following steps:
s21: when the time domain motion vector prediction is carried out based on the sub-blocks, the current block is divided according to the size of the current block, and a plurality of sub-blocks with the same size and the side length larger than a preset value are obtained.
In one embodiment, partitioning the current block to obtain a plurality of sub-blocks with the same size includes: and dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and the side length larger than a preset value. Compared with the prior art, the method has the advantages that the cross segmentation is directly carried out on the current block, the segmentation mode is flexibly selected according to the size of the current block, and the prediction accuracy can be effectively improved under the condition of controlling the complexity and increasing the amplitude.
The rest of the step S21 is substantially the same as the step S11 in the above embodiment, and is not repeated here.
S22: a temporal motion vector for each sub-block is obtained.
In one embodiment, obtaining the temporal motion vector of each sub-block comprises: and acquiring a time domain motion vector corresponding to a pixel block at any position of each sub-block, and taking the time domain motion vector corresponding to the pixel block as the time domain motion vector of each sub-block. The specific content of step S22 is substantially the same as that of step S22 in the above embodiment, and is not repeated here.
In other embodiments, the pixel block at the fixed corner of each sub-block may also be selected, and the temporal motion vector of the pixel block is obtained and used as the temporal motion vector of the sub-block.
S23: and respectively carrying out motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
In an embodiment, motion compensation and distortion cost calculation are performed on the temporal motion vectors of all sub-blocks of the current block, instead of performing motion compensation and distortion cost calculation using the original TMVP mode, so that a more accurate calculation result can be obtained.
Referring to fig. 9, another embodiment of the present application provides a temporal motion vector prediction apparatus 30, which includes a current block partitioning module 31 and an obtaining module 32, so as to implement the temporal motion vector prediction method according to the above-mentioned corresponding embodiment. Specifically, when performing temporal motion vector prediction based on sub-blocks, the current block segmentation module 31 segments the current block to obtain a plurality of sub-blocks with the same size. The obtaining module 32 obtains a time domain motion vector corresponding to a pixel block at any position in each sub-block, and determines the time domain motion vector of each sub-block based on the time domain motion vector corresponding to the pixel block. The positions of the sub-blocks of the device 30 for acquiring the time domain motion vector are more flexible, and the positions of different sub-blocks for acquiring the time domain motion vector can be different, so that the accuracy of acquiring the time domain motion vector by the sub-blocks is improved, and the accuracy of predicting the time domain motion vector is further improved.
Referring to fig. 10, another embodiment of the present application provides a temporal motion vector prediction apparatus 40, which includes a current block partitioning module 41 and an obtaining module 42, so as to implement the temporal motion vector prediction method according to the above-mentioned corresponding embodiment. Specifically, when the current block partitioning module 41 performs temporal motion vector prediction based on sub-blocks, the current block is partitioned according to the size of the current block, and a plurality of sub-blocks with the same size and the side length greater than a predetermined value are obtained. The acquisition module 42 acquires a temporal motion vector of each sub-block. The device 40 flexibly selects the segmentation mode according to the size of the current block, and can effectively improve the prediction accuracy under the condition of controlling the complexity increase amplitude.
Referring to fig. 11, another embodiment of the present application provides an electronic device 50, which includes a memory 51 and a processor 52 coupled to each other, wherein the processor 52 is configured to execute program instructions stored in the memory 51 to implement the temporal motion vector prediction method according to any of the embodiments described above. In one particular implementation scenario, electronic device 50 may include, but is not limited to: a microcomputer, a server, and the electronic device 50 may also include a mobile device such as a notebook computer, a tablet computer, and the like, which is not limited herein.
In particular, the processor 52 is configured to control itself and the memory 51 to implement the frame reference method of any of the above embodiments. Processor 52 may also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip having signal processing capabilities. The Processor 52 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 52 may be commonly implemented by an integrated circuit chip.
Referring to fig. 12, another embodiment of the present application provides a computer-readable storage medium 60, on which program data 61 is stored, and when the program data 61 is executed by a processor, the temporal motion vector prediction method of any of the embodiments is implemented.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely one type of logical division, and an actual implementation may have another division, for example, a unit or a component may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on network elements. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium 60. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium 60 and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method of the embodiments of the present application. And the aforementioned storage medium 60 includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings, or which are directly or indirectly applied to other related technical fields, are intended to be included within the scope of the present application.

Claims (12)

1. A temporal motion vector prediction method, the method comprising:
when the time domain motion vector prediction is carried out based on the sub-blocks, the current block is divided to obtain a plurality of sub-blocks with the same size;
and acquiring a time domain motion vector corresponding to a pixel block at any position in each sub-block, and determining the time domain motion vector of each sub-block based on the time domain motion vector corresponding to the pixel block.
2. The method of claim 1, wherein obtaining the temporal motion vector corresponding to the pixel block at any position in each of the sub-blocks, and determining the temporal motion vector of each of the sub-blocks based on the temporal motion vector corresponding to the pixel block comprises:
and acquiring a corresponding time domain motion vector by utilizing a pixel block at the position of the upper left corner, the lower left corner, the upper right corner, the lower right corner or the central point of each sub-block so as to determine the time domain motion vector of each sub-block, wherein all the side lengths of the pixel block are four pixels.
3. The method of claim 1, further comprising:
responding to the current sub-block being unable to obtain a valid time domain motion vector;
determining a valid temporal motion vector of the current sub-block based on a temporal motion vector of a block of pixels of another location in the current sub-block; alternatively, the first and second electrodes may be,
obtaining effective time domain motion vectors of the rest sub-blocks in the current block in a first sequence, and taking the effective time domain motion vectors of the rest sub-blocks in the previous sequence as the time domain motion vectors of the current sub-block; alternatively, the first and second electrodes may be,
and obtaining the effective time domain motion vectors of the first number of the rest sub-blocks in the current block in a second sequence or randomly, and taking the effective time domain motion vectors as the effective time domain motion vectors of the current sub-block after weighted averaging.
4. The method of claim 1, wherein the partitioning the current block into sub-blocks of equal size comprises:
and dividing the current block according to the size of the current block to obtain a plurality of sub-blocks with the same size and the side length larger than a preset value.
5. The method of claim 4, wherein the partitioning the current block according to the size of the current block comprises:
comparing lengths of a first side and a second side of the current block, the first side being adjacent to the second side;
in response to that the length ratio of the first side edge to the second side edge is greater than or equal to a first predetermined multiple, evenly dividing the current block in a direction perpendicular to the first side edge, wherein the first predetermined multiple is greater than one; alternatively, the first and second electrodes may be,
in response to the length ratio of the second side to the first side being greater than or equal to a second predetermined multiple, evenly dividing the current block in a direction perpendicular to the second side, the second predetermined multiple being greater than one; alternatively, the first and second electrodes may be,
in response to the first side being equal to the second side, equally dividing the current block in a cross-partition method.
6. The method of claim 4, wherein the partitioning the current block according to the size of the current block comprises:
acquiring the lengths of a first side edge and a second side edge of the current block, wherein the first side edge is adjacent to the second side edge;
in response to the length of the first side being at least twice the predetermined value, equally dividing the current block in a direction perpendicular to the first side; and/or the presence of a gas in the gas,
equally dividing the current block in a direction perpendicular to the second side in response to the length of the second side being at least twice the predetermined value.
7. The method of claim 1, further comprising, prior to said partitioning the current block:
acquiring two first time domain motion vectors corresponding to a first group of pixel blocks at the diagonal position of the current block and two second time domain motion vectors corresponding to a second group of pixel blocks at the diagonal position of the current block;
judging whether the two first time domain motion vectors are the same and/or whether the two second time domain motion vectors are the same;
if so, the current block is not partitioned.
8. The method of claim 4, wherein the temporal motion vector prediction based on sub-blocks comprises:
judging whether the current prediction mode is a skip/direct inter-frame prediction mode, whether the side lengths of a first side and a second side of the current block are both greater than or equal to the preset value, at least one side is greater than or equal to two times of the preset value, and the first side is adjacent to the second side;
and if so, performing time domain motion vector prediction based on the sub-blocks.
9. The method of claim 1, further comprising:
and performing motion compensation and distortion cost calculation on the time domain motion vectors of all the sub-blocks of the current block.
10. A temporal motion vector prediction method, the method comprising:
when time domain motion vector prediction is carried out based on sub-blocks, the current block is divided according to the size of the current block, and a plurality of sub-blocks with the same size and the side length larger than a preset value are obtained;
and acquiring a time domain motion vector of each sub-block.
11. An electronic device comprising a memory and a processor coupled to each other, the processor being configured to execute program instructions stored in the memory to implement the prediction method of any one of claims 1 to 9 or the prediction method of claim 10.
12. A computer-readable storage medium on which program data are stored, which program data, when being executed by a processor, carry out the prediction method of any one of claims 1 to 9 or the prediction method of claim 10.
CN202010784588.XA 2020-08-06 2020-08-06 Time domain motion vector prediction method, device and storage medium Active CN112055207B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010784588.XA CN112055207B (en) 2020-08-06 2020-08-06 Time domain motion vector prediction method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010784588.XA CN112055207B (en) 2020-08-06 2020-08-06 Time domain motion vector prediction method, device and storage medium

Publications (2)

Publication Number Publication Date
CN112055207A true CN112055207A (en) 2020-12-08
CN112055207B CN112055207B (en) 2024-05-31

Family

ID=73601488

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010784588.XA Active CN112055207B (en) 2020-08-06 2020-08-06 Time domain motion vector prediction method, device and storage medium

Country Status (1)

Country Link
CN (1) CN112055207B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060059770A (en) * 2004-11-29 2006-06-02 엘지전자 주식회사 Method and apparatus for deriving motion vectors of macro blocks from motion vectors of pictures of base layer when encoding/decoding video signal
CN109089119A (en) * 2017-06-13 2018-12-25 浙江大学 A kind of method and apparatus of motion-vector prediction
WO2019109955A1 (en) * 2017-12-08 2019-06-13 华为技术有限公司 Interframe prediction method and apparatus, and terminal device
CN110545424A (en) * 2019-08-21 2019-12-06 浙江大华技术股份有限公司 Inter-frame prediction method based on MMVD (multimedia MediaVision video) mode, video coding method, related device and equipment
CN110876282A (en) * 2018-07-02 2020-03-10 华为技术有限公司 Motion vector prediction method and related device
CN111316639A (en) * 2018-04-09 2020-06-19 腾讯美国有限责任公司 Method and apparatus for sub-block motion vector prediction
WO2020151274A1 (en) * 2019-01-23 2020-07-30 华为技术有限公司 Image display order determination method and apparatus, and video coding/decoding device

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060059770A (en) * 2004-11-29 2006-06-02 엘지전자 주식회사 Method and apparatus for deriving motion vectors of macro blocks from motion vectors of pictures of base layer when encoding/decoding video signal
CN109089119A (en) * 2017-06-13 2018-12-25 浙江大学 A kind of method and apparatus of motion-vector prediction
WO2019109955A1 (en) * 2017-12-08 2019-06-13 华为技术有限公司 Interframe prediction method and apparatus, and terminal device
CN111316639A (en) * 2018-04-09 2020-06-19 腾讯美国有限责任公司 Method and apparatus for sub-block motion vector prediction
CN110876282A (en) * 2018-07-02 2020-03-10 华为技术有限公司 Motion vector prediction method and related device
WO2020151274A1 (en) * 2019-01-23 2020-07-30 华为技术有限公司 Image display order determination method and apparatus, and video coding/decoding device
CN110545424A (en) * 2019-08-21 2019-12-06 浙江大华技术股份有限公司 Inter-frame prediction method based on MMVD (multimedia MediaVision video) mode, video coding method, related device and equipment

Also Published As

Publication number Publication date
CN112055207B (en) 2024-05-31

Similar Documents

Publication Publication Date Title
CN110581999B (en) Chroma decoder side motion vector refinement
JP7358436B2 (en) Motion vector refinement for multi-reference prediction
CN110620928B (en) Multiple candidates of different precision
CN111937391B (en) Video processing method and apparatus for sub-block motion compensation in video codec systems
TWI663872B (en) Method and apparatus of motion refinement based on bi-directional optical flow for video coding
US20230093366A1 (en) Multi- iteration motion vector refinement
CN110677675B (en) Method, device and storage medium for efficient affine Merge motion vector derivation
TWI729402B (en) Weighted interweaved prediction
CN113056914B (en) Partial position based difference calculation
CN112913249A (en) Simplified coding and decoding of generalized bi-directional prediction index
JP2019519995A (en) Video Coding Using Adaptive Motion Information Improvement
CN110677674B (en) Method, apparatus and non-transitory computer-readable medium for video processing
CN112292862A (en) Memory access window and padding for motion vector modification and motion compensation
CN112292861B (en) Sub-pixel accurate correction method based on error surface for decoding end motion vector correction
CN110740327A (en) Motion compensation of overlapping blocks
CN111201795A (en) Memory access window and padding for motion vector modification
CN110719467B (en) Prediction method of chrominance block, encoder and storage medium
CN110876063A (en) Fast coding method for interleaving prediction
CN112055207B (en) Time domain motion vector prediction method, device and storage medium
CN110876064B (en) Partially interleaved prediction
CN110557639B (en) Application of interleaved prediction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant