WO2022116574A1 - 基于二次多项式的运动矢量预测方法、系统及计算机介质 - Google Patents

基于二次多项式的运动矢量预测方法、系统及计算机介质 Download PDF

Info

Publication number
WO2022116574A1
WO2022116574A1 PCT/CN2021/109738 CN2021109738W WO2022116574A1 WO 2022116574 A1 WO2022116574 A1 WO 2022116574A1 CN 2021109738 W CN2021109738 W CN 2021109738W WO 2022116574 A1 WO2022116574 A1 WO 2022116574A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
motion
quadratic polynomial
motion vector
model
Prior art date
Application number
PCT/CN2021/109738
Other languages
English (en)
French (fr)
Inventor
罗伟节
杨光芒
刘星宇
滕波
Original Assignee
浙江智慧视频安防创新中心有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江智慧视频安防创新中心有限公司 filed Critical 浙江智慧视频安防创新中心有限公司
Publication of WO2022116574A1 publication Critical patent/WO2022116574A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding

Definitions

  • the present application belongs to the technical field of video coding and decoding, and in particular, relates to a motion vector prediction method, system and computer medium based on a quadratic polynomial.
  • Inter-frame prediction based on motion estimation is a key technology in video coding, and is widely used in various coding standards, including H.263, H.264, H.265, VCC, MPEG4 and other coding standards.
  • prediction coding is usually performed on the motion vector of inter-frame prediction.
  • a motion vector predictor obtained by predicting a motion vector, such as the Alternative temporal motion vector prediction (ATMVP) optional temporal motion vector prediction supported by the latest H.266 standard.
  • ATMVP Alternative temporal motion vector prediction
  • each coding unit CU is allowed to obtain multiple sets of motion information from multiple blocks of collocated reference pictures that are smaller than the size of the current coding unit CU.
  • the optional temporal motion vector prediction ATMVP improves the temporal motion vector prediction TMVP by obtaining a set of multiple motion information from blocks smaller than the current coding unit CU, where the motion information includes a motion vector and a reference index. Specifically, it includes two steps: 1. Identifying a block corresponding to a temporal vector in a reference image.
  • the reference picture is called the motion source picture: the reference picture and the corresponding block are determined by the motion information of the spatial neighboring blocks of the current CU. To avoid the repeated scanning process of adjacent blocks, the first candidate in the merge candidate list of the current CU is used.
  • the sub-CU can be an NxN block (N is 4 by default).
  • N is 4 by default.
  • the motion information of its corresponding block (the smallest motion grid covering the center sample) is used to derive the sub-CU's motion information. After identifying the motion information of the corresponding NxN block, it is converted into the motion vector and reference index of the current sub-CU.
  • the existing coding technology only uses linear weighted combination of multiple prediction vectors as the prediction result of the temporal motion vector.
  • the motion prediction accuracy decreases, the motion vector residual is large, and the coding efficiency is low.
  • the present invention proposes a motion vector prediction method, system and computer medium based on a quadratic polynomial, aiming to solve the problem of the accuracy of motion prediction and the improvement of the accuracy of motion prediction when the motion trajectory of the coding block changes in a curve in the inter-frame prediction of the existing coding technology.
  • a motion vector prediction method based on a quadratic polynomial is provided, and video encoding and decoding is performed according to the obtained motion vector prediction value, which specifically includes the following steps:
  • the motion vector prediction value is obtained
  • Video encoding and decoding is performed based on motion vector predictors.
  • the current motion block, the first forward matching block and the second forward matching block respectively belong to three consecutive frames of images.
  • a quadratic polynomial motion model is constructed, and the model parameters of the quadratic polynomial motion model are determined according to the pixel positions at the same position in the current motion block, the first forward matching block, and the second forward matching block. , including:
  • the prediction reference block of the current coding block is determined according to the quadratic polynomial motion model for determining the model parameters, which specifically includes:
  • the current coding block is set as the forward motion block of the prediction reference block, and the horizontal distance between the current coding block and the prediction reference block is a fixed value;
  • the position coordinates of the prediction reference block are determined according to the position coordinates of the current coding block, the lateral distance between the current coding block and the prediction reference block, and the quadratic polynomial motion model.
  • the prediction reference block of the current coding block is determined according to the quadratic polynomial motion model for determining the model parameters, which specifically includes:
  • the prediction reference block is set to be the forward motion block of the current coding block, and the horizontal distance between the current coding block and the prediction reference block is a fixed value;
  • the position coordinates of the prediction reference block are determined according to the position coordinates of the current coding block, the lateral distance between the current coding block and the prediction reference block, and the quadratic polynomial motion model.
  • the method further includes:
  • the prediction reference block is determined to be the final prediction reference block.
  • the method further includes:
  • a new prediction reference block is determined according to the quadratic polynomial motion model for determining the model parameters.
  • a motion vector prediction system based on a quadratic polynomial which specifically includes:
  • Motion estimation unit used to sequentially determine the current motion block, the first forward matching block and the second forward matching block in reverse chronological order through block-based motion estimation;
  • Quadratic polynomial motion model unit used to construct a quadratic polynomial motion model, and determine the model of the quadratic polynomial motion model according to the pixel positions of the current motion block, the first forward matching block and the second forward matching block at the same position parameter;
  • Prediction reference block unit used to determine the prediction reference block of the current coding block according to the quadratic polynomial motion model for determining the model parameters;
  • Motion vector prediction unit used to obtain the motion vector prediction value according to the prediction reference block
  • Video codec unit used for video codec based on motion vector predictors.
  • a motion vector prediction device based on a quadratic polynomial including:
  • memory used to store executable instructions
  • Processor for connecting with the memory to execute executable instructions to complete the motion vector prediction method based on quadratic polynomial.
  • a computer-readable storage medium on which a computer program is stored; the computer program is executed by a processor to implement a quadratic polynomial-based motion vector prediction method.
  • the current motion block, the first forward matching block, and the second forward matching block are sequentially determined in reverse time order.
  • Matching block build a quadratic polynomial motion model, and determine the model parameters of the quadratic polynomial motion model according to the current motion block, the first forward matching block, and the pixel position at the same position in the second forward matching block; determine the model parameters according to The quadratic polynomial motion model of , determines the prediction reference block of the current coding block; and obtains the motion vector prediction value according to the prediction reference block.
  • video coding and decoding is performed according to the obtained motion vector prediction value, which improves the motion prediction accuracy and coding efficiency when the motion trajectory of the coding block changes in a curve.
  • FIG. 1 shows a schematic diagram of steps of a method for predicting a motion vector based on a quadratic polynomial according to an embodiment of the present application
  • Fig. 2 shows a schematic diagram of the position of the coding block in the motion vector prediction method based on the quadratic polynomial according to an embodiment of the present application
  • FIG. 3 shows another schematic diagram of the location of the coding block in the motion vector prediction method based on the quadratic polynomial according to an embodiment of the present application
  • FIG. 4 shows a schematic structural diagram of a quadratic polynomial-based motion vector prediction system according to an embodiment of the present application
  • FIG. 5 shows a schematic structural diagram of a motion vector prediction device based on a quadratic polynomial according to an embodiment of the present application.
  • the inventor found that in the coding process, since the motion vector of a motion block has a certain correlation and predictability with several forward matching blocks in the time domain, in order to improve the compression efficiency, usually Predictive encoding is performed on the inter-predicted motion vector.
  • the existing coding technology only uses a linear weighted combination of multiple prediction vectors as a prediction method for temporal motion vectors.
  • the motion vector residual is large by the same motion prediction method, which leads to a significant drop in prediction accuracy and low coding efficiency.
  • the present application establishes a quadratic polynomial as a motion model, performs training through several images, obtains parameters of the quadratic polynomial, and determines the matching block position in the subsequent image frame through the quadratic polynomial motion model. Therefore, when the motion trajectory of the coding block changes in a curve, the motion prediction accuracy and coding efficiency are improved.
  • the quadratic polynomial-based motion vector prediction method, system, and computer medium of the present application determine the current motion block, the first forward matching block, and the second forward matching block sequentially in reverse time sequence through block-based motion estimation.
  • block construct a quadratic polynomial motion model, and determine the model parameters of the quadratic polynomial motion model according to the current motion block, the first forward matching block, and the pixel position at the same position in the second forward matching block;
  • the quadratic polynomial motion model determines the prediction reference block of the current coding block; according to the prediction reference block, the motion vector prediction value is obtained.
  • video encoding and decoding is performed according to the obtained motion vector prediction value.
  • FIG. 1 shows a schematic diagram of steps of a motion vector prediction method based on a quadratic polynomial according to an embodiment of the present application.
  • the second-order polynomial-based motion vector prediction method specifically includes the following steps:
  • S101 Determine the current motion block, the first forward matching block, and the second forward matching block sequentially in time sequence through block-based motion estimation.
  • FIG. 2 and FIG. 3 show schematic diagrams of the positions of coding blocks in the motion vector prediction method based on the quadratic polynomial according to the embodiment of the present application.
  • inter-frame prediction mainly searches for matching blocks through block-based motion estimation.
  • the matching blocks may be searched in I or P frames that appear before, or in image frames that appear later.
  • the current motion block, the first forward matching block and the second forward matching block can be sequentially determined, that is, the forward matching block of the current motion block and the forward matching block of the current motion block need to be determined.
  • the forward matching blocks are labeled as motion block 1, motion block 2, and motion block 3, respectively.
  • the motion blocks 1, 2 and 3 belong to three different image frames, and the image frames are reconstructed images.
  • the current motion block, the first forward matching block and the second forward matching block belong to three consecutive frames of images respectively.
  • the first three image frames are continuously encoded in the manner specified by H.264, H.265, H.266 and subsequent standards, and at least three frames of images are formed and kept in the cache. In one embodiment, the three frames of images are continuously.
  • S102 Construct a quadratic polynomial motion model, and determine model parameters of the quadratic polynomial motion model according to pixel positions at the same position in the current motion block, the first forward matching block, and the second forward matching block.
  • S102 specifically includes:
  • the current pixel point, the first forward pixel point and the second forward pixel point are substituted into the quadratic polynomial motion model, and the model parameters of the quadratic polynomial motion model are obtained by calculating and solving.
  • the coordinates of the first pixel in the upper left corner of the current motion block, the first forward matching block and the second forward matching block are p0(x0,y0), p1(x1,y1) and p3( x2, y2), then these three points can form a curve as shown in Figure 2.
  • the curve can be represented by a quadratic polynomial, that is, to construct a quadratic polynomial motion model, formula (1) is:
  • the first forward matching is obtained by processing according to other existing conventional methods. block and a second forward matching block.
  • S103 Determine the prediction reference block of the current coding block according to the quadratic polynomial motion model for which the model parameters are determined.
  • S103 specifically includes:
  • the position coordinates of the prediction reference block are determined according to the position coordinates of the current coding block, the lateral distance between the current coding block and the prediction reference block, and the quadratic polynomial motion model.
  • the current coding block is the forward motion block of the prediction reference block, or the prediction reference block is set as the forward motion block of the current coding block.
  • the position of the current coding block is substituted into a quadratic polynomial motion model to solve the position of the prediction reference block.
  • the motion blocks 3, 2, and 1 correspond to points P3, P1, and P0, and the position of P4 (x3, y3) in motion block 4 that most likely matches it in the next frame can be predicted.
  • the motion block where P4 is located is the motion block that motion blocks 3, 2 and 1 may match. Conversely, for this motion block 4, the best forward matching block is motion block 3.
  • the position of the current coding block is also substituted into the quadratic polynomial motion model to obtain the position of the prediction reference block.
  • the position of the point P3 (x2, y2) in the motion block 2 and the motion vector prediction can be predicted.
  • x2 is substituted into the second
  • y a*x ⁇ 2+b*x+c
  • the best matching block of the current block is determined through the motion block where the horizontal and vertical coordinates (x2, y2) are located, that is, the prediction reference block is determined.
  • the positions of the motion blocks in the two frames of images are compared to obtain a motion vector.
  • the method further includes:
  • Calculate the total difference between the current coding block and the prediction reference block for example, it can be the sum of the absolute values of interpolation of each pixel value
  • the search is terminated, and the current prediction reference block is determined to be the final prediction reference block.
  • a new prediction reference block is determined according to the quadratic polynomial motion model for determining the model parameters. Continue to search for candidate reference blocks in other locations until a prediction reference block that satisfies the condition is found. Motion vector predictors are then calculated. In this method, the candidate matching block calculated by binomial is actually used as the starting point of the matching search, which is beneficial to reduce the search workload.
  • S105 Perform video coding and decoding according to the motion vector predictor.
  • the forward matching block of the forward matching block is determined according to a similar method. As shown in FIG. 3 , if the motion block 4 is the current coding block, it is the block 3 and the block 2 respectively. Then the weighted average of the motion vectors of block 3 and block 2 is taken as the motion vector predictor of the current coded block.
  • the processing flow of the encoding end first determine the motion vector prediction value according to the above S101-S104, then subtract the motion prediction from the actual motion vector to form the motion vector residual, and then convert it into The video stream is decoded and played by the decoder (device).
  • the process of encoding and converting the motion vector into a video stream belongs to the conventional technology of video encoding, which will not be repeated here in this application.
  • the decoding end In the processing flow of the decoding end, first determine the motion vector prediction value according to the above S101-S104, and then obtain the motion vector residual from the video bit rate through entropy decoding, inverse quantization, inverse transformation and other processing procedures.
  • the reconstructed image block is obtained by adding the motion vector residual to the prediction reference block.
  • the process of reconstructing the motion block belongs to the conventional technology of video decoding, which is not repeated here in this application.
  • the motion vector prediction method based on quadratic polynomial in the embodiment of the present application, through block-based motion estimation, in reverse time order, the current motion block, the first forward matching block and the second forward matching block are sequentially determined;
  • the second-order polynomial motion model determines the model parameters of the second-order polynomial motion model according to the pixel positions of the same position in the current motion block, the first forward matching block and the second forward matching block; according to the second-order polynomial motion that determines the model parameters
  • the model determines the prediction reference block of the current coding block; according to the prediction reference block, the motion vector prediction value is obtained.
  • video coding and decoding is performed according to the obtained motion vector prediction value, which improves the motion prediction accuracy and coding efficiency when the motion trajectory of the coding block changes in a curve.
  • This embodiment provides a motion vector prediction system based on quadratic polynomials.
  • the motion vector prediction system based on quadratic polynomials in this embodiment please refer to the motion vector based on quadratic polynomials in other embodiments.
  • the specific implementation content of the vector prediction method please refer to the motion vector based on quadratic polynomials in other embodiments.
  • Fig. 4 shows a schematic structural diagram of a motion vector prediction system based on a quadratic polynomial according to an embodiment of the present application.
  • the motion vector prediction system based on the quadratic polynomial in the embodiment of the present application specifically includes a motion estimation unit 10, a quadratic polynomial motion model unit 20, a prediction reference block unit 30, a motion vector prediction unit 40, and a video encoder.
  • Decoding unit 50 specifically includes a motion estimation unit 10, a quadratic polynomial motion model unit 20, a prediction reference block unit 30, a motion vector prediction unit 40, and a video encoder.
  • Decoding unit 50 Decoding unit 50.
  • Motion estimation unit 10 configured to sequentially determine the current motion block, the first forward matching block and the second forward matching block in reverse time sequence through block-based motion estimation.
  • FIG. 2 and FIG. 3 show schematic diagrams of the positions of coding blocks in the motion vector prediction method based on the quadratic polynomial according to the embodiment of the present application.
  • inter-frame prediction mainly searches for matching blocks through block-based motion estimation.
  • the matching blocks may be searched in I or P frames that appear before, or in image frames that appear later.
  • the current motion block, the first forward matching block and the second forward matching block can be sequentially determined, that is, the forward matching block of the current motion block and the forward matching block of the current motion block need to be determined.
  • the forward matching blocks are labeled as motion block 1, motion block 2, and motion block 3, respectively.
  • the motion blocks 1, 2 and 3 belong to three different image frames, and the image frames are reconstructed images.
  • the current motion block, the first forward matching block and the second forward matching block belong to three consecutive frames of images respectively.
  • the first three image frames are continuously encoded in the manner specified by H.264, H.265, H.266 and subsequent standards, and at least three frames of images are formed and kept in the cache. In one embodiment, the three frames of images are continuously.
  • the quadratic polynomial motion model unit 20 is used for constructing a quadratic polynomial motion model, and determines the position of the quadratic polynomial motion model according to the pixel position at the same position in the current motion block, the first forward matching block and the second forward matching block. model parameters.
  • the current pixel point, the first forward pixel point and the second forward pixel point are substituted into the quadratic polynomial motion model, and the model parameters of the quadratic polynomial motion model are obtained by calculating and solving.
  • the coordinates of the first pixel in the upper left corner of the current motion block, the first forward matching block and the second forward matching block are p0(x0,y0), p1(x1,y1) and p3( x2, y2), then these three points can form a curve as shown in Figure 2.
  • the curve can be represented by a quadratic polynomial, that is, to construct a quadratic polynomial motion model, formula (1) is:
  • the first forward matching is obtained by processing according to other existing conventional methods. block and a second forward matching block.
  • Prediction reference block unit 30 configured to determine the prediction reference block of the current coding block according to the quadratic polynomial motion model for determining the model parameters.
  • the position coordinates of the prediction reference block are determined according to the position coordinates of the current coding block, the lateral distance between the current coding block and the prediction reference block, and the quadratic polynomial motion model.
  • the current coding block is the forward motion block of the prediction reference block, or the prediction reference block is set as the forward motion block of the current coding block.
  • the position of the current coding block is substituted into a quadratic polynomial motion model to solve the position of the prediction reference block.
  • the motion blocks 3, 2, and 1 correspond to points P3, P1, and P0, and the position of P4 (x3, y3) in motion block 4 that most likely matches it in the next frame can be predicted.
  • the motion block where P4 is located is the motion block that motion blocks 3, 2 and 1 may match. Conversely, for this motion block 4, the best forward matching block is motion block 3.
  • the position of the current coding block is also substituted into the quadratic polynomial motion model to solve the position of the prediction reference block.
  • the position of the point P3 (x2, y2) in the motion block 2 and the motion vector prediction can be predicted.
  • x2 is substituted into the second
  • y a*x ⁇ 2+b*x+c
  • the best matching block of the current block is determined through the motion block where the horizontal and vertical coordinates (x2, y2) are located, that is, the prediction reference block is determined.
  • the position of the motion block in the two frames of images is compared to obtain the motion vector.
  • the method further includes:
  • Calculate the total difference between the current coding block and the prediction reference block for example, it can be the sum of the absolute values of interpolation of each pixel value
  • the search is terminated, and the current prediction reference block is determined as the final prediction reference block.
  • a new prediction reference block is determined according to the quadratic polynomial motion model for determining the model parameters. Continue to search for candidate reference blocks in other locations until a prediction reference block that satisfies the condition is found. The motion vector predictors are then calculated. In this method, the candidate matching block calculated by binomial is actually used as the starting point of the matching search, which is beneficial to reduce the search workload.
  • Motion vector prediction unit 40 used to obtain a motion vector prediction value according to the prediction reference block.
  • the video codec unit 50 performs video codec according to the motion vector predictor.
  • the forward matching block of the forward matching block is determined according to a similar method. As shown in FIG. 3 , if the motion block 4 is the current coding block, it is the block 3 and the block 2 respectively. Then the weighted average of the motion vectors of block 3 and block 2 is taken as the motion vector predictor of the current coded block.
  • the motion estimation unit 10 the quadratic polynomial motion model unit 20, the prediction reference block unit 30 and the motion vector prediction unit 40 are used to determine the motion vector prediction value, and then the actual motion vector is subtracted from the motion prediction, The motion vector residual is formed, and then converted into a video code stream through the processes of transformation, quantization, entropy coding, etc. for the decoding end (device) to decode and play.
  • the process of encoding and converting the motion vector into a video stream belongs to the conventional technology of video encoding, which will not be repeated here in this application.
  • the motion vector prediction value is first determined by the motion estimation unit 10, the quadratic polynomial motion model unit 20, the prediction reference block unit 30 and the motion vector prediction unit 40, and then, from the video bit rate, through entropy decoding, Inverse quantization, inverse transformation and other processing procedures to obtain motion vector residuals.
  • the reconstructed image block is obtained by adding the motion vector residual to the prediction reference block.
  • the process of reconstructing the motion block belongs to the conventional technology of video decoding, which is not repeated here in this application.
  • the motion estimation unit 10 sequentially determines the current motion block, the first forward matching block and the second forward matching block in reverse time order through block-based motion estimation block; the quadratic polynomial motion model unit 20 constructs a quadratic polynomial motion model, and determines the model of the quadratic polynomial motion model according to the pixel position at the same position in the current motion block, the first forward matching block and the second forward matching block parameters; the prediction reference block unit 30 determines the prediction reference block of the current coding block according to the quadratic polynomial motion model that determines the model parameters; the motion vector prediction unit 40 obtains the motion vector prediction value according to the prediction reference block. Finally, the video codec unit 50 performs video codec according to the motion vector predictor. It is realized that the accuracy of motion prediction and the coding efficiency are improved when the motion trajectory of the coding block changes in a curve.
  • This embodiment provides a motion vector prediction device based on quadratic polynomials.
  • the motion vector prediction device based on quadratic polynomials in this embodiment please refer to the motion vector based on quadratic polynomials in other embodiments.
  • the specific implementation content of the vector prediction method or system please refer to the motion vector based on quadratic polynomials in other embodiments.
  • FIG. 5 shows a schematic structural diagram of a second-order polynomial-based motion vector prediction apparatus 400 according to an embodiment of the present application.
  • the motion vector prediction device 400 includes:
  • memory 402 for storing executable instructions
  • Processor 401 for connecting with the memory 402 to execute executable instructions to complete the motion vector prediction method.
  • the schematic diagram 5 is only an example of the motion vector prediction apparatus 400, and does not constitute a limitation on the motion vector prediction apparatus 400, and may include more or less components than those shown in the figure, or combine some components, Or different components, such as the motion vector prediction device 400, may also include input and output devices, network access devices, buses, and the like.
  • the so-called processor 401 can also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits (Application Specific Integrated Circuits, ASICs), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor can be a microprocessor or the processor 401 can also be any conventional processor, etc.
  • the processor 401 is the control center of the motion vector prediction device 400, and uses various interfaces and lines to connect the entire motion vector prediction device various parts.
  • the memory 402 can be used to store computer-readable instructions, and the processor 401 implements various functions of the motion vector prediction apparatus 400 by running or executing the computer-readable instructions or modules stored in the memory 402 and calling the data stored in the memory 402. .
  • the memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required for at least one function, and the like;
  • the motion vector prediction apparatus 400 uses the created data and the like.
  • the memory 402 may include a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card (Flash Card), at least one disk storage device, a flash memory devices, Read-Only Memory (ROM), Random Access Memory (RAM), or other non-volatile/volatile storage devices.
  • SMC Smart Media Card
  • SD Secure Digital
  • Flash Card Flash Card
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • the integrated modules of the motion vector prediction apparatus 400 are implemented in the form of software functional modules and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present invention realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium, The computer-readable instructions, when executed by the processor, can implement the steps of the above-mentioned various method embodiments.
  • This embodiment provides a computer-readable storage medium on which a computer program is stored; the computer program is executed by a processor to implement the quadratic polynomial-based motion vector prediction method in other embodiments.
  • the current motion block, the first forward matching block and the second forward matching block are sequentially determined in reverse time sequence through block-based motion estimation ; Construct a quadratic polynomial motion model, and determine the model parameters of the quadratic polynomial motion model according to the pixel positions at the same position in the current motion block, the first forward matching block and the second forward matching block; A second degree polynomial motion model is used to determine the prediction reference block of the current coding block; according to the prediction reference block, the motion vector prediction value is obtained. Finally, video coding and decoding is performed according to the obtained motion vector prediction value, which improves the motion prediction accuracy and coding efficiency when the motion trajectory of the coding block changes in a curve.
  • the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
  • computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions
  • the apparatus implements the functions specified in the flow or flows of the flowcharts and/or the block or blocks of the block diagrams.
  • first, second, third, etc. may be used in the present invention to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present invention.
  • word "if” as used herein may be interpreted as "at the time of” or "when” or "in response to determining.”

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例中提供了一种基于二次多项式的运动矢量预测方法、系统及计算机介质，通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；根据预测参考块，得到运动矢量预测值。最后根据得到的运动矢量预测值进行视频编解码，实现了在面对编码块运动轨迹呈曲线变化时，提高了运动预测准确度以及编码效率。

Description

基于二次多项式的运动矢量预测方法、系统及计算机介质 技术领域
本申请属于视频编解码技术领域，具体地，涉及一种基于二次多项式的运动矢量预测方法、系统及计算机介质。
背景技术
基于运动估计的帧间预测是视频编码中的关键技术，被广泛应用到各种编解码标准中，包括H.263，H.264，H.265，VCC，MPEG4等编码标准。编码过程中，由于一个运动块的运动矢量与其时域上的若干前向匹配块存在一定的相关性和可预测性，因此为了提高压缩效率，通常还对帧间预测的运动矢量进行预测编码。
对于运动矢量进行预测得到的预测运动矢量(motion vector predictor，MVP)，例如最新的H.266标准支持的Alternative temporal motion vector prediction(ATMVP)可选时域运动矢量预测。在可选时域运动矢量预测ATMVP中，允许每个编码单元CU从并置的参考图像的多个小于当前编码单元CU大小的块中，得到多个运动信息集合。
可选时域运动矢量预测ATMVP通过从比当前编码单元CU小的块中，获取多个运动信息的集合改善了时域运动矢量预测TMVP，其中，运动信息包括了运动矢量和参考索引。具体的，包括两个步骤：一、识别一个参考图像中的时域矢量对应块。参考图像被称为运动源图像：参考图像和对应块由当前CU的空间相邻块的运动信息确定。为了避免相邻块的重复扫描过程，使用当前CU的merge候选列表中的第一个候选。将第一个可用的运动矢量及其相关参考索引设置为运动源图像的时间向量和索引，例如子CU可以为NxN块(N默认为4)。二、将当前CU划分为子CU，获取每个子CU对应的MV以及参考索引：通过运动源图像中的时间向量，将当前CU的时间矢量添加到当前CU的坐标中的方式，来识别子CU的对应块。对于每个子CU，其对应块的运动信息(覆盖中心样本的最小运动网格)用于导出子CU的运动信息。在识别对应的NxN块的运动信息之后，将其转换成当前的子CU的运动矢量和参考索引。
由此可见，现有的编码技术只采用了对多个预测向量进行线性加权组合作为时域运动矢量的预测结果。当运动物体运动轨迹呈曲线变化时，运动预测准确度下降，运动矢量残差较大，编码效率较低。
发明内容
本发明提出了一种基于二次多项式的运动矢量预测方法、系统及计算机介质，旨在解决现有编码技术在帧间预测中，面对编码块运动轨迹呈曲线变化时，运动预测准确度以及编码效率低的问题。
根据本申请实施例的第一个方面，提供了一种基于二次多项式的运动矢量预测方法，根据得到的运动矢量预测值进行视频编解码，具体包括以下步骤：
通过基于块的运动估计，按时间顺序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；
构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；
根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；
根据预测参考块，得到运动矢量预测值；
根据运动矢量预测值进行视频编解码。
在本申请一些实施方式中，当前运动块、第一前向匹配块以及第二前向匹配块分别属于连续的三帧图像。
在本申请一些实施方式中，构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数，具体包括：
确定当前运动块、第一前向匹配块以及第二前向匹配块中左上角第一个像素，分别为当前像素点、第一前向像素点以及第二前向像素点；
将当前像素点、第一前向像素点以及第二前向像素点代入二次多项式运动模型，通过计算求解，得到二次多项式运动模型的模型参数。
在本申请一些实施方式中，根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块，具体包括：
设定当前编码块为预测参考块的前向运动块，当前编码块与预测参考块的横向距离为一固定值；
根据当前编码块的位置坐标、当前编码块与预测参考块的横向距离以及二次多项式运动模型，确定预测参考块的位置坐标。
在本申请一些实施方式中，根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块，具体包括：
设定预测参考块为当前编码块的前向运动块，当前编码块与预测参考块的横 向距离为一固定值；
根据当前编码块的位置坐标、当前编码块与预测参考块的横向距离以及二次多项式运动模型，确定预测参考块的位置坐标。
在本申请一些实施方式中，根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块之后，还包括：
计算当前编码块与预测参考块之间的总差值；
当总差值在小于或等于阈值时，则确定预测参考块为最终的预测参考块。
在本申请一些实施方式中，根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块之后，还包括：
计算当前编码块与预测参考块之间的总差值；
当总差值在大于阈值时，则根据确定模型参数的二次多项式运动模型，确定新的预测参考块。
根据本申请实施例的第二个方面，提供了一种基于二次多项式的运动矢量预测系统，具体包括：
运动估计单元：用于通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；
二次多项式运动模型单元：用于构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；
预测参考块单元：用于根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；
运动矢量预测单元：用于根据预测参考块，得到运动矢量预测值；
视频编解码单元：用于根据运动矢量预测值进行视频编解码。
根据本申请实施例的第三个方面，提供了一种基于二次多项式的运动矢量预测设备，包括：
存储器：用于存储可执行指令；以及
处理器:用于与存储器连接以执行可执行指令从而完成基于二次多项式的运动矢量预测方法。
根据本申请实施例的第四个方面，提供了一种计算机可读存储介质，其上存储有计算机程序；计算机程序被处理器执行以实现基于二次多项式的运动矢量预 测方法。
采用本申请实施例中的基于二次多项式的运动矢量预测方法、系统及计算机介质，通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；根据预测参考块，得到运动矢量预测值。最后根据得到的运动矢量预测值进行视频编解码，实现了在面对编码块运动轨迹呈曲线变化时，提高了运动预测准确度以及编码效率。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1中示出了根据本申请实施例的基于二次多项式的运动矢量预测方法的步骤示意图；
图2中示出了根据本申请实施例的基于二次多项式的运动矢量预测方法中编码块的位置示意图；
图3中示出了根据本申请实施例的基于二次多项式的运动矢量预测方法中编码块的另一位置示意图；
图4中示出了根据本申请实施例的基于二次多项式的运动矢量预测系统的结构示意图；
图5中示出了根据本申请实施例的基于二次多项式的运动矢量预测设备的结构示意图。
具体实施方式
在实现本申请的过程中，发明人发现编码过程中，由于一个运动块的运动矢量与其时域上的若干前向匹配块存在一定的相关性和可预测性，因此为了提高压缩效率，通常还对帧间预测的运动矢量进行预测编码。
然而，现有编码技术只采用了对多个预测向量进行线性加权组合作为时域运动矢量的预测方法。但是，当运动物体运动轨迹呈曲线变化时，通过相同的运动预测方法，运动矢量残差较大，导致了预测准确度大幅下降，编码效率较低。
鉴于以上问题，本申请建立二次多项式作为运动模型，通过若干图像进行训练，得到二次多项式的参数，通过二次多项式运动模型进而确定后续图像帧中的匹配块位置。从而在面对编码块运动轨迹呈曲线变化时，提高了运动预测准确度以及编码效率。
具体的，本申请的基于二次多项式的运动矢量预测方法、系统及计算机介质，通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；根据预测参考块，得到运动矢量预测值。最后根据得到的运动矢量预测值进行视频编解码。
为了使本申请实施例中的技术方案及优点更加清楚明白，以下结合附图对本申请的示例性实施例进行进一步详细的说明，显然，所描述的实施例仅是本申请的一部分实施例，而不是所有实施例的穷举。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
实施例1
图1中示出了根据本申请实施例的基于二次多项式的运动矢量预测方法的步骤示意图。
如图1所示，本申请实施例的基基于二次多项式的运动矢量预测方法，具体包括以下步骤：
S101：通过基于块的运动估计，按时间顺序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块。
图2和图3中示出了根据本申请实施例的基于二次多项式的运动矢量预测方法中编码块的位置示意图。
目前，帧间预测主要通过基于块的运动估计来搜索匹配块，匹配块可能在之前出现的I或P帧中，也可以在之后出现的图像帧中搜索。
如图2所示，按时间顺序，则可以依次确定当前运动块、第一前向匹配块以及第二前向匹配块，即需要确定当前运动块的前向匹配块、以及前向匹配块的前向匹配块，分别标记为运动块1、运动块2和运动块3。
运动块1、2和3分属不同的三个图像帧中，该图像帧为重构图像。
在一种实施方式中，当前运动块、第一前向匹配块以及第二前向匹配块分别属于连续的三帧图像。最初的三个图像帧按H.264、H.265、H.266及后续标准规定的方式继续编码，并形成至少3帧图像留着缓存中，在一种实施例中，这三帧图像是连续的。
S102：构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数。
S102具体包括：
首先，确定当前运动块、第一前向匹配块以及第二前向匹配块中左上角第一个像素，分别为当前像素点、第一前向像素点以及第二前向像素点；
然后，将当前像素点、第一前向像素点以及第二前向像素点代入二次多项式运动模型，通过计算求解，得到二次多项式运动模型的模型参数。
在具体实施过程中，假设当前运动块、第一前向匹配块以及第二前向匹配块左上角第一个像素的坐标分别为p0(x0,y0)、p1(x1,y1)和p3(x2,y2)，那么这三个点可以构成一条如图2所示的一条曲线。该曲线可以用一个二次多项式表示，即构建二次多项式运动模型公式(1)为：
y＝a*x^2+b*x+c             公式(1)
将三个像素点坐标位置p0(x0,y0)、p1(x1,y1)和p3(x2,y2)，代入公式(1)，得到以下公式(2)：
y0＝a*x0^2+b*x0+c；
y1＝a*x1^2+b*x1+c；
y2＝a*x2^2+b*x2+c；              公式(2)
根据公式(2)，进行方程式求解，可以分别求得二次多项式运动模型的模型参数a、b以及c。
再通过运动估计预测前向匹配块的过程中，若存在当前块不支持运动预测、或前向匹配块不支持前向预测的情况时，则按照其它现有常规方法处理得到第一前向匹配块以及第二前向匹配块。
S103：根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块。
S103具体包括：
设定当前编码块与预测参考块的横向距离为一固定值；
根据当前编码块的位置坐标、当前编码块与预测参考块的横向距离以及二次多项式运动模型，确定预测参考块的位置坐标。
其中，当前编码块为预测参考块的前向运动块，或者设定预测参考块为当前编码块的前向运动块。
具体实施过程中，当前编码块为预测参考块的前向运动块时，将当前编码块位置代入二次多项式运动模型，来求解预测参考块的位置。
例如，如图2和3所示，由运动块3、2以及1对应P3、P1以及P0点，可以预测下一帧中与其最可能匹配的运动块4中P4(x3,y3)的位置。以运动块3确定预测参考块为例，运动块3与运动块4的横向距离为一固定值，即设定运动块4中P4的横坐标为x3＝x2+1，那么将x3代入二次多项式运动模型y＝a*x^2+b*x+c中，得到运动块4中P4的竖坐标为y3＝a*(x2+1)^2+(x2+1)*x+c，进而通过横坐标和竖坐标(x3,y3)，确定预测参考块。
如图2和3所示，P4所在运动块即运动块3、2以及1可能匹配的运动块。反过来说，对于该运动块4，最佳前向匹配块是运动块3。
另一具体实施过程中，当前编码块为预测参考块的前向运动块时，同样将当前编码块位置代入二次多项式运动模型，来求解预测参考块的位置。
例如，如图2和3所示，由运动块4中点P4(x3,y3)，可以预测运动块2中点P3(x2,y2)的位置及运动矢量预测。以运动块4确定预测参考块为例，运动块4与运动块3的横向距离为一固定值，即设定运动块3中P3的横坐标为x2＝x3-1，那么将x2代入二次多项式运动模型y＝a*x^2+b*x+c中，得到运动块3中P3的竖坐标为y2＝a*(x3-1)^2+b*(x3-1)+c，进而通过横坐标和竖坐标(x2,y2)所在运动块，确定当前块的最佳匹配块，即确定预测参考块。
在通过S103找到最佳匹配块，即预测参考块后，比较运动块在两帧图像中的位置，得到运动矢量。
在本申请一些实施方式中，根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块之后，还包括：
计算当前编码块与预测参考块之间的总差值，例如，可以是各像素值插值绝对值之和；
当总差值在小于或等于阈值时，则搜索中止，确定当前预测参考块为最终的预测参考块。当总差值在大于阈值时，则根据确定模型参数的二次多项式运动模 型，确定新的预测参考块。继续搜索其它位置的候选参考块，直到找到满足条件的预测参考块为止。然后计算运动矢量预测值。通过这种方法中，实际上将通过二项式计算出来的候选匹配块作为匹配搜索的起点，有利于减少搜索工作量。
S104：根据预测参考块，得到运动矢量预测值。
S105：根据运动矢量预测值进行视频编解码。
在一种实施例中，在找到当前编码块的前向匹配块后，按照类似方法确定该前向匹配块的前向匹配块。如图3所示，若运动块4为当前编码块，则分别为块3和块2。然后将块3和块2的运动矢量做加权平均，作为当前编码块的运动矢量预测值。
编码端处理流程中，首先按照以上S101-S104，确定运动矢量预测值，之后，将实际运动矢量减去运动预测，形成运动矢量残差，然后再通过经过变换、量化、熵编码等过程转换为视频码流供解码端(器)解码播放。将运动矢量编码转换为视频码流的这一过程属于视频编码常规技术，本申请此处不再赘述。
解码端的处理流程中，首先按照以上S101-S104，确定运动矢量预测值，之后，从视频码率中，经过熵解码、反量化、反变换等处理流程，得到运动矢量残差。将运动矢量残差与预测参考块相加，即得到重建图像块。重构运动块这一过程属于视频解码常规技术，本申请此处不再赘述。
采用本申请实施例中的基于二次多项式的运动矢量预测方法，通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；根据预测参考块，得到运动矢量预测值。最后根据得到的运动矢量预测值进行视频编解码，实现了在面对编码块运动轨迹呈曲线变化时，提高了运动预测准确度以及编码效率。
实施例2
本实施例提供了一种基于二次多项式的运动矢量预测系统，对于本实施例的基于二次多项式的运动矢量预测系统中未披露的细节，请参照其它实施例中的基于二次多项式的运动矢量预测方法的具体实施内容。
图4中示出了根据本申请实施例的基于二次多项式的运动矢量预测系统的 结构示意图。
如图4所示，本申请实施例的基于二次多项式的运动矢量预测系统，具体包括运动估计单元10、二次多项式运动模型单元20、预测参考块单元30、运动矢量预测单元40以及视频编解码单元50。
运动估计单元10：用于通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块。
图2和图3中示出了根据本申请实施例的基于二次多项式的运动矢量预测方法中编码块的位置示意图。
目前，帧间预测主要通过基于块的运动估计来搜索匹配块，匹配块可能在之前出现的I或P帧中，也可以在之后出现的图像帧中搜索。
如图2所示，按时间顺序，则可以依次确定当前运动块、第一前向匹配块以及第二前向匹配块，即需要确定当前运动块的前向匹配块、以及前向匹配块的前向匹配块，分别标记为运动块1、运动块2和运动块3。
运动块1、2和3分属不同的三个图像帧中，该图像帧为重构图像。
在一种实施方式中，当前运动块、第一前向匹配块以及第二前向匹配块分别属于连续的三帧图像。最初的三个图像帧按H.264、H.265、H.266及后续标准规定的方式继续编码，并形成至少3帧图像留着缓存中，在一种实施例中，这三帧图像是连续的。
二次多项式运动模型单元20：用于构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数。
具体包括：
首先，确定当前运动块、第一前向匹配块以及第二前向匹配块中左上角第一个像素，分别为当前像素点、第一前向像素点以及第二前向像素点；
然后，将当前像素点、第一前向像素点以及第二前向像素点代入二次多项式运动模型，通过计算求解，得到二次多项式运动模型的模型参数。
在具体实施过程中，假设当前运动块、第一前向匹配块以及第二前向匹配块左上角第一个像素的坐标分别为p0(x0,y0)、p1(x1,y1)和p3(x2,y2)，那么这三个点可以构成一条如图2所示的一条曲线。该曲线可以用一个二次多项式表示，即构建二次多项式运动模型公式(1)为：
y＝a*x^2+b*x+c，             公式(1)
将三个像素点坐标位置p0(x0,y0)、p1(x1,y1)和p3(x2,y2)，代入公式(1)，得到以下公式(2)：
y0＝a*x0^2+b*x0+c；
y1＝a*x1^2+b*x1+c；
y2＝a*x2^2+b*x2+c；         公式(2)
根据公式(2)，进行方程式求解，可以分别求得二次多项式运动模型的模型参数a、b以及c。
再通过运动估计预测前向匹配块的过程中，若存在当前块不支持运动预测、或前向匹配块不支持前向预测的情况时，则按照其它现有常规方法处理得到第一前向匹配块以及第二前向匹配块。
预测参考块单元30：用于根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块。
具体包括：
设定当前编码块与预测参考块的横向距离为一固定值；
根据当前编码块的位置坐标、当前编码块与预测参考块的横向距离以及二次多项式运动模型，确定预测参考块的位置坐标。
其中，当前编码块为预测参考块的前向运动块，或者设定预测参考块为当前编码块的前向运动块。
具体实施过程中，当前编码块为预测参考块的前向运动块时，将当前编码块位置代入二次多项式运动模型，来求解预测参考块的位置。
例如，如图2和3所示，由运动块3、2以及1对应P3、P1以及P0点，可以预测下一帧中与其最可能匹配的运动块4中P4(x3,y3)的位置。以运动块3确定预测参考块为例，运动块3与运动块4的横向距离为一固定值，即设定运动块4中P4的横坐标为x3＝x2+1，那么将x3代入二次多项式运动模型y＝a*x^2+b*x+c中，得到运动块4中P4的竖坐标为y3＝a*(x2+1)^2+(x2+1)*x+c，进而通过横坐标和竖坐标(x3,y3)，确定预测参考块。
如图2和3所示，P4所在运动块即运动块3、2以及1可能匹配的运动块。反过来说，对于该运动块4，最佳前向匹配块是运动块3。
另一具体实施过程中，当前编码块为预测参考块的前向运动块时，同样将当 前编码块位置代入二次多项式运动模型，来求解预测参考块的位置。
例如，如图2和3所示，由运动块4中点P4(x3,y3)，可以预测运动块2中点P3(x2,y2)的位置及运动矢量预测。以运动块4确定预测参考块为例，运动块4与运动块3的横向距离为一固定值，即设定运动块3中P3的横坐标为x2＝x3-1，那么将x2代入二次多项式运动模型y＝a*x^2+b*x+c中，得到运动块3中P3的竖坐标为y2＝a*(x3-1)^2+b*(x3-1)+c，进而通过横坐标和竖坐标(x2,y2)所在运动块，确定当前块的最佳匹配块，即确定预测参考块。
在找到最佳匹配块，即预测参考块后，比较运动块在两帧图像中的位置，得到运动矢量。
在本申请一些实施方式中，根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块之后，还包括：
计算当前编码块与预测参考块之间的总差值，例如，可以是各像素值插值绝对值之和；
当总差值在小于或等于阈值时，则搜索中止，确定当前预测参考块为最终的预测参考块。当总差值在大于阈值时，则根据确定模型参数的二次多项式运动模型，确定新的预测参考块。继续搜索其它位置的候选参考块，直到找到满足条件的预测参考块为止。然后计算运动矢量预测值。通过这种方法中，实际上将通过二项式计算出来的候选匹配块作为匹配搜索的起点，有利于减少搜索工作量。
运动矢量预测单元40：用于根据预测参考块，得到运动矢量预测值。视频编解码单元50根据运动矢量预测值进行视频编解码。
在一种实施例中，在找到当前编码块的前向匹配块后，按照类似方法确定该前向匹配块的前向匹配块。如图3所示，若运动块4为当前编码块，则分别为块3和块2。然后将块3和块2的运动矢量做加权平均，作为当前编码块的运动矢量预测值。
编码端处理流程中，首先通过运动估计单元10、二次多项式运动模型单元20、预测参考块单元30以及运动矢量预测单元40，确定运动矢量预测值，之后，将实际运动矢量减去运动预测，形成运动矢量残差，然后再通过经过变换、量化、熵编码等过程转换为视频码流供解码端(器)解码播放。将运动矢量编码转换为视频码流的这一过程属于视频编码常规技术，本申请此处不再赘述。
解码端的处理流程中，首先通过运动估计单元10、二次多项式运动模型单 元20、预测参考块单元30以及运动矢量预测单元40确定运动矢量预测值，之后，从视频码率中，经过熵解码、反量化、反变换等处理流程，得到运动矢量残差。将运动矢量残差与预测参考块相加，即得到重建图像块。重构运动块这一过程属于视频解码常规技术，本申请此处不再赘述。
采用本申请实施例中的基于二次多项式的运动矢量预测系统，运动估计单元10通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；二次多项式运动模型单元20构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；预测参考块单元30根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；运动矢量预测单元40根据预测参考块，得到运动矢量预测值。最后视频编解码单元50根据运动矢量预测值进行视频编解码。实现了在面对编码块运动轨迹呈曲线变化时，提高了运动预测准确度以及编码效率。
实施例3
本实施例提供了一种基于二次多项式的运动矢量预测设备，对于本实施例的基于二次多项式的运动矢量预测设备中未披露的细节，请参照其它实施例中的基于二次多项式的运动矢量预测方法或系统具体的实施内容。
图5中示出了根据本申请实施例的基于二次多项式的运动矢量预测设备400的结构示意图。
如图5所示，运动矢量预测设备400，包括：
存储器402：用于存储可执行指令；以及
处理器401:用于与存储器402连接以执行可执行指令从而完成运动矢量预测方法。
本领域技术人员可以理解，示意图5仅仅是运动矢量预测设备400的示例，并不构成对运动矢量预测设备400的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如运动矢量预测设备400还可以包括输入输出设备、网络接入设备、总线等。
所称处理器401(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器401也可以是任何常规的处理器等，处理器401是运动矢量预测设备400的控制中心，利用各种接口和线路连接整个运动矢量预测设备400的各个部分。
存储器402可用于存储计算机可读指令，处理器401通过运行或执行存储在存储器402内的计算机可读指令或模块，以及调用存储在存储器402内的数据，实现运动矢量预测设备400的各种功能。存储器402可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如声音播放功能、图像播放功能等)等；存储数据区可存储根据运动矢量预测设备400使用所创建的数据等。此外，存储器402可以包括硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)或其他非易失性/易失性存储器件。
运动矢量预测设备400集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实现上述实施例方法中的全部或部分流程，也可以通过计算机可读指令来指令相关的硬件来完成，的计算机可读指令可存储于一计算机可读存储介质中，该计算机可读指令在被处理器执行时，可实现上述各个方法实施例的步骤。
实施例4
本实施例提供了一种计算机可读存储介质，其上存储有计算机程序；计算机程序被处理器执行以实现其他实施例中的基于二次多项式的运动矢量预测方法。
采用本申请实施例中的基于二次多项式的运动矢量预测设备及计算机介质，通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；构建二次多项式运动模型，根据当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定二次多项式运动模型的模型参数；根据确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；根据预测参考块，得到运动矢量预测值。最后根据得到的运动矢量预测值进行视频编解码，实现了在面对编码块运动轨迹呈曲线变化时，提高了运动预测 准确度以及编码效率。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在本发明使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本发明。在本发明和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本发明可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本发明范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可 以被解释成为“在……时”或“当……时”或“响应于确定”。
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。
显然，本领域的技术人员可以对本申请进行各种改动和变型而不脱离本申请的精神和范围。这样，倘若本申请的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (10)

  1. 一种基于二次多项式的运动矢量预测方法，其特征在于，具体包括以下步骤：
    通过基于块的运动估计，按时间顺序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；
    构建二次多项式运动模型，根据所述当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定所述二次多项式运动模型的模型参数；
    根据所述确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；
    根据所述预测参考块，得到当前编码块的运动矢量预测值；
    根据所述运动矢量预测值进行视频编解码。
  2. 根据权利要求1所述的基于二次多项式的运动矢量预测方法，其特征在于，所述当前运动块、第一前向匹配块以及第二前向匹配块属于连续的三帧图像。
  3. 根据权利要求1所述的基于二次多项式的运动矢量预测方法，其特征在于，所述构建二次多项式运动模型，根据所述当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定所述二次多项式运动模型的模型参数，具体包括：
    确定当前运动块、第一前向匹配块以及第二前向匹配块中左上角第一个像素，分别为当前像素点、第一前向像素点以及第二前向像素点；
    将所述当前像素点、第一前向像素点以及第二前向像素点代入所述二次多项式运动模型，通过计算求解，得到所述二次多项式运动模型的模型参数。
  4. 根据权利要求1所述的基于二次多项式的运动矢量预测方法，其特征在于，所述根据所述确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块，具体包括：
    设定当前编码块为所述预测参考块的前向运动块，所述当前编码块与预测参考块的横向距离为一固定值；
    根据所述当前编码块的位置坐标、当前编码块与预测参考块的横向距离以及二次多项式运动模型，确定所述预测参考块的位置坐标。
  5. 根据权利要求1所述的基于二次多项式的运动矢量预测方法，其特征在于， 所述根据所述确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块，具体包括：
    设定预测参考块为所述当前编码块的前向运动块，所述当前编码块与预测参考块的横向距离为一固定值；
    根据所述当前编码块的位置坐标、当前编码块与预测参考块的横向距离以及二次多项式运动模型，确定所述预测参考块的位置坐标。
  6. 根据权利要求1所述的基于二次多项式的运动矢量预测方法，其特征在于，所述根据所述确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块之后，还包括：
    计算所述当前编码块与预测参考块之间的总差值；
    当总差值在小于或等于阈值时，则确定所述预测参考块为最终的预测参考块。
  7. 根据权利要求1所述的基于二次多项式的运动矢量预测方法，其特征在于，所述根据所述确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块之后，还包括：
    计算所述当前编码块与预测参考块之间的总差值；
    当总差值在大于阈值时，则根据所述确定模型参数的二次多项式运动模型，确定新的预测参考块。
  8. 一种基于二次多项式的运动矢量预测系统，其特征在于，具体包括：
    运动估计单元：用于通过基于块的运动估计，按时间倒序，依次确定当前运动块、第一前向匹配块以及第二前向匹配块；
    二次多项式运动模型单元：用于构建二次多项式运动模型，根据所述当前运动块、第一前向匹配块以及第二前向匹配块中相同位置的像素点位置，确定所述二次多项式运动模型的模型参数；
    预测参考块单元：用于根据所述确定模型参数的二次多项式运动模型，确定当前编码块的预测参考块；
    运动矢量预测单元：用于根据所述预测参考块，得到运动矢量预测值；
    视频编解码单元：用于根据所述运动矢量预测值进行视频编解码。
  9. 一种基于二次多项式的运动矢量预测设备，其特征在于，包括：
    存储器：用于存储可执行指令；以及
    处理器:用于与所述存储器连接以执行所述可执行指令从而完成权利要求 1-7任一项所述的基于二次多项式的运动矢量预测方法。
  10. 一种计算机可读存储介质，其特征在于，其上存储有计算机程序；计算机程序被处理器执行以实现如权利要求1-7任一项所述的基于二次多项式的运动矢量预测方法。
PCT/CN2021/109738 2020-12-04 2021-07-30 基于二次多项式的运动矢量预测方法、系统及计算机介质 WO2022116574A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011397791.8A CN112203091B (zh) 2020-12-04 2020-12-04 基于二次多项式的运动矢量预测方法、系统及计算机介质
CN202011397791.8 2020-12-04

Publications (1)

Publication Number Publication Date
WO2022116574A1 true WO2022116574A1 (zh) 2022-06-09

Family

ID=74033812

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/109738 WO2022116574A1 (zh) 2020-12-04 2021-07-30 基于二次多项式的运动矢量预测方法、系统及计算机介质

Country Status (2)

Country Link
CN (1) CN112203091B (zh)
WO (1) WO2022116574A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112203091B (zh) * 2020-12-04 2021-05-04 浙江智慧视频安防创新中心有限公司 基于二次多项式的运动矢量预测方法、系统及计算机介质
CN113518225B (zh) * 2021-08-10 2023-06-16 辽宁师范大学 6次多项式拟合匹配误差曲面的快速缩放运动估计方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070291844A1 (en) * 2006-06-16 2007-12-20 International Business Machines Corporation Method and system for non-linear motion estimaftion
CN102461150A (zh) * 2009-04-23 2012-05-16 想象技术有限公司 在运动估计系统中使用动量和加速度矢量的对象跟踪
CN103647972A (zh) * 2008-12-03 2014-03-19 株式会社日立制作所 运动图像解码方法和运动图像编码方法
US20150208082A1 (en) * 2014-01-21 2015-07-23 Vixs Systems, Inc. Video encoder with reference picture prediction and methods for use therewith
CN112203091A (zh) * 2020-12-04 2021-01-08 浙江智慧视频安防创新中心有限公司 基于二次多项式的运动矢量预测方法、系统及计算机介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100389601C (zh) * 2005-10-09 2008-05-21 北京中星微电子有限公司 一种视频电子防抖的装置
US10271062B2 (en) * 2016-03-18 2019-04-23 Google Llc Motion vector prediction through scaling
WO2019089382A1 (en) * 2017-10-31 2019-05-09 Vid Scale, Inc. 360-degree video coding using face-based geometry padding

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070291844A1 (en) * 2006-06-16 2007-12-20 International Business Machines Corporation Method and system for non-linear motion estimaftion
CN103647972A (zh) * 2008-12-03 2014-03-19 株式会社日立制作所 运动图像解码方法和运动图像编码方法
CN102461150A (zh) * 2009-04-23 2012-05-16 想象技术有限公司 在运动估计系统中使用动量和加速度矢量的对象跟踪
US20150208082A1 (en) * 2014-01-21 2015-07-23 Vixs Systems, Inc. Video encoder with reference picture prediction and methods for use therewith
CN112203091A (zh) * 2020-12-04 2021-01-08 浙江智慧视频安防创新中心有限公司 基于二次多项式的运动矢量预测方法、系统及计算机介质

Also Published As

Publication number Publication date
CN112203091A (zh) 2021-01-08
CN112203091B (zh) 2021-05-04

Similar Documents

Publication Publication Date Title
JP5579937B2 (ja) 低複雑度の動きベクトルを導出するシステム及び方法
JP4928627B2 (ja) 動き推定の探索範囲の適応選択方法及び装置
WO2019191890A1 (zh) 用于图像处理的方法和图像处理装置
JP2019115061A (ja) 符号化装置、符号化方法、復号装置、復号方法、及びプログラム
US10412409B2 (en) Encoding system using motion estimation and encoding method using motion estimation
CN108134939B (zh) 一种运动估计方法及装置
WO2022116574A1 (zh) 基于二次多项式的运动矢量预测方法、系统及计算机介质
JP7330243B2 (ja) 低減されたメモリアクセスを用いてｆｒｕｃモードでビデオデータを符号化又は復号する方法及び装置
WO2020140915A1 (zh) 视频处理方法和装置
JP5306485B2 (ja) 動きベクトル予測符号化方法、動きベクトル予測復号方法、動画像符号化装置、動画像復号装置およびそれらのプログラム
WO2015010319A1 (zh) 一种基于p帧的多假设运动补偿编码方法
CN108419082B (zh) 一种运动估计方法及装置
WO2011099440A1 (ja) 動きベクトル予測符号化方法、動きベクトル予測復号方法、動画像符号化装置、動画像復号装置およびそれらのプログラム
CN102215387B (zh) 视频图像处理方法以及编/解码器
US11863783B2 (en) Artificial intelligence-based image encoding and decoding apparatus and method
KR100490730B1 (ko) 비디오 영상 압축 시스템에서의 적응적 패턴을 이용한움직임 예측 장치 및 방법
JP2012147291A (ja) 画像符号化装置、画像符号化方法及びプログラム、画像復号装置、画像復号方法及びプログラム
WO2019128443A1 (zh) 一种图像解码方法及解码器
CN114402616A (zh) 当前块的预测方法及装置、设备、存储介质
WO2022021310A1 (zh) 编码方法、装置、计算处理设备、计算机程序及存储介质
JP2019530299A (ja) 画像コーディング／デコーディング方法、装置及び画像処理機器
WO2009128208A1 (ja) 動画像符号化装置、動画像復号化装置、動画像符号化方法、および動画像復号化方法
CN105306953A (zh) 图像编码方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21899613

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21899613

Country of ref document: EP

Kind code of ref document: A1