CN113873257B - Method, device and equipment for constructing motion information candidate list - Google Patents

Method, device and equipment for constructing motion information candidate list Download PDF

Info

Publication number
CN113873257B
CN113873257B CN202111153145.1A CN202111153145A CN113873257B CN 113873257 B CN113873257 B CN 113873257B CN 202111153145 A CN202111153145 A CN 202111153145A CN 113873257 B CN113873257 B CN 113873257B
Authority
CN
China
Prior art keywords
motion information
current block
time domain
block
backward
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111153145.1A
Other languages
Chinese (zh)
Other versions
CN113873257A (en
Inventor
曹小强
陈方栋
王莉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN202111153145.1A priority Critical patent/CN113873257B/en
Publication of CN113873257A publication Critical patent/CN113873257A/en
Application granted granted Critical
Publication of CN113873257B publication Critical patent/CN113873257B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures

Abstract

The application provides a method, a device and equipment for constructing a motion information candidate list, wherein the method comprises the following steps: if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame; determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block; and constructing a motion information candidate list of the current block based on the time domain motion information of the current block. The prediction accuracy is improved through the application.

Description

Method, device and equipment for constructing motion information candidate list
Technical Field
The present application relates to the field of encoding and decoding, and in particular, to a method, an apparatus, and a device for constructing a motion information candidate list.
Background
In order to achieve the purpose of saving space, video images are transmitted after being encoded, and the complete video encoding can include the processes of prediction, transformation, quantization, entropy encoding, filtering and the like. For the prediction process, the prediction process may include intra-frame prediction and inter-frame prediction, where inter-frame prediction refers to using the correlation of the video time domain to predict the current pixel using the pixels adjacent to the coded image, so as to achieve the purpose of effectively removing the video time domain redundancy. The intra-frame prediction means that the current pixel is predicted by using the pixel of the coded block of the current frame image by using the correlation of a video spatial domain so as to achieve the purpose of removing the video spatial domain redundancy.
In the inter-frame prediction process, a motion information candidate list is usually required to be constructed for the current block, the motion information candidate list may include a plurality of candidate motion information, and the candidate motion information in the motion information candidate list may be spatial motion information or temporal motion information. Obviously, in order to construct the motion information candidate list, the temporal motion information of the current block needs to be acquired, and the temporal motion information acquired in the related art has the problems of poor prediction effect, poor coding performance and the like.
Disclosure of Invention
The application provides a method, a device and equipment for constructing a motion information candidate list, and the prediction accuracy is improved.
The application provides a method for constructing a motion information candidate list, which comprises the following steps:
if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame;
determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block; and constructing a motion information candidate list of the current block based on the time domain motion information of the current block.
The application provides a device for constructing a motion information candidate list, which comprises:
the determining module is used for determining a time domain adjacent block of the current block from a same-position frame of the current frame if the current frame where the current block is located is a B frame; determining temporal motion information of the current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; wherein if the forward motion information of the time domain neighboring block is unavailable and the backward motion information of the time domain neighboring block is available, the determining module determines the backward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block; and the constructing module is used for constructing a motion information candidate list of the current block based on the time domain motion information of the current block.
The application provides a decoding side device, including: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor;
the processor is configured to execute machine executable instructions to perform the steps of:
if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame;
determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block; and constructing a motion information candidate list of the current block based on the time-domain motion information of the current block.
The application provides a coding end device, including: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor;
the processor is configured to execute machine executable instructions to perform the steps of:
if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame;
determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block; and constructing a motion information candidate list of the current block based on the time domain motion information of the current block.
According to the technical scheme, when the forward motion information of the time domain adjacent blocks is unavailable and the backward motion information of the time domain adjacent blocks is available, the backward motion information in the time domain motion information of the current block can be determined according to the backward motion information of the time domain adjacent blocks, so that the time domain motion information of the current block is obtained, the accuracy of prediction is improved, the prediction performance is improved, the coding performance is improved, the predicted value of the current block is closer to the original pixel, and the coding performance is improved.
Drawings
FIG. 1 is a schematic diagram of a video coding framework;
FIG. 2 is a flow chart of a method for constructing a motion information candidate list according to an embodiment of the present application;
3A-3C are schematic diagrams of temporal motion information derivation according to an embodiment of the present application;
FIGS. 4A-4C are schematic diagrams of weight prediction angles in one embodiment of the present application;
FIG. 4D is a diagram illustrating weighted prediction locations in one embodiment of the present application;
FIG. 4E is a graph illustrating reference weight values in one embodiment of the present application;
FIG. 4F is a diagram of neighboring blocks of a current block in one embodiment of the present application;
FIG. 4G is a schematic view of an angular sector in one embodiment of the present application;
FIG. 5A is a diagram illustrating a UMVE mode prediction process in one embodiment of the present application;
FIG. 5B is a schematic diagram of an AFFINE mode prediction process according to an embodiment of the present application;
fig. 6A is a schematic structural diagram of a motion information candidate list constructing apparatus according to an embodiment of the present application;
fig. 6B is a hardware structure diagram of a decoding-side device in an embodiment of the present application;
fig. 6C is a hardware structure diagram of an encoding end device in an embodiment of the present application.
Detailed Description
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples and claims of this application, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items. It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, depending on the context, without departing from the scope of embodiments of the present application. Further, the word "if" as used may be interpreted as "at \8230; \8230when", or "when 8230; \8230, when", or "in response to a determination".
The embodiment of the application provides a method, a device and equipment for constructing a motion information candidate list, and relates to the following concepts:
a video coding framework: referring to fig. 1, a video encoding frame may be used to implement the processing flow of the encoding end in the embodiment of the present application, a schematic diagram of a video decoding frame is similar to that in fig. 1, and details are not repeated here, and a video decoding frame may be used to implement the processing flow of the decoding end in the embodiment of the present application. Illustratively, in the video encoding framework and the video decoding framework, there may be included, but not limited to: intra prediction/inter prediction, motion estimation/motion compensation, reference picture buffer, in-loop filtering, reconstruction, transformation, quantization, inverse transformation, inverse quantization, entropy encoder, etc. At the encoding end, the processing flow at the encoding end can be realized through the matching of the modules, and at the decoding end, the processing flow at the decoding end can be realized through the matching of the modules.
Intra prediction (intra prediction): in view of the fact that strong spatial correlation exists between adjacent blocks in an image, the current uncoded block can be predicted by using the reconstructed surrounding pixels as reference pixels, and therefore, only the residual signal (original signal-prediction signal) needs to be subjected to subsequent coding processing instead of the original signal, so that the redundancy in the spatial domain is effectively removed, the purpose of removing the spatial domain redundancy of the video is achieved, and the compression efficiency of the video signal is greatly improved.
Inter prediction (inter prediction): based on the correlation of the video time domain, because a video sequence contains stronger time domain correlation, the pixels of the current image are predicted by using the pixels of the adjacent coded images, and the aim of effectively removing the video time domain redundancy is fulfilled.
And (3) transformation: the image described in the form of pixels in the spatial domain is converted into the transform domain and represented by transform coefficients. Because most images contain more flat areas and slowly-changed areas, the conversion can convert the dispersed distribution of image energy in a space domain into the relatively concentrated distribution of a conversion domain, remove the frequency domain correlation among signals, and can effectively compress a code stream by matching with a quantization process.
Entropy coding: a series of symbols of elements representing a video sequence are transformed into a binary code stream for transmission or storage, in a lossless coding manner based on the principle of information entropy, and the input symbols may include quantized transform coefficients, motion vector information, prediction mode information, transform quantization related syntax, and the like. And outputting the final code stream of the compressed original video. Entropy coding can effectively remove the statistical redundancy of the video element symbols, and is one of the important tools for ensuring the compression efficiency of video coding.
Motion Vector (MV): in inter prediction, a relative displacement between a current block of a current frame and a reference block of a reference frame may be represented using a motion vector. Each divided block has a corresponding motion vector transmitted to a decoding side, and if the motion vector of each block is independently encoded and transmitted, especially a large number of blocks of small size, a lot of bits are consumed. In order to reduce the number of bits used to encode a motion vector, the spatial correlation between adjacent blocks can be used to predict the motion vector of a current block from the motion vectors of adjacent encoded blocks, and then the prediction difference can be encoded, which can effectively reduce the number of bits representing the motion vector. When encoding a Motion Vector of a current block, the Motion Vector of the current block may be predicted using Motion vectors of adjacent encoded blocks, and then a Difference value (MVD) between a Prediction value (MVP) of the Motion Vector Prediction and a true estimate value of the Motion Vector may be encoded.
Motion Information (Motion Information): since the motion vector indicates a position offset between the current block and a certain reference block, in order to accurately acquire information pointing to the block, index information of a reference frame image is required in addition to the motion vector to indicate which reference frame image the current block uses. In video coding techniques, a reference frame picture list may be generally established for a current frame, and the reference frame picture index information indicates that the current block uses the next reference frame picture in the reference frame picture list.
For example, many coding techniques also support multiple reference picture lists, and therefore, an index value, which may be referred to as a reference direction, may also be used to indicate which reference picture list is used. In summary, in the video encoding technology, motion-related information such as motion vectors, reference frame indexes, reference directions, etc. may be collectively referred to as motion information.
Skip mode (Skip mode) and Direct mode (Direct mode): in the inter-frame prediction process, because a video has strong time-domain correlation, that is, two adjacent frames of images have many similar blocks, the current block of the current frame is often subjected to motion search in the adjacent reference images, and a block which is most matched with the current block is found as a reference block. Because the similarity between the reference block and the current block is high and the difference between the reference block and the current block is very small, the code rate overhead of encoding the difference is usually much smaller than the code rate overhead brought by encoding the pixel value of the current block.
In order to indicate the position of the block that most matches the current block, it is necessary to encode a lot of motion information to be transmitted to the decoding end so that the decoding end knows the position of the most matching block. And the motion information, especially the motion vector, needs to consume a very large code rate to be transmitted. In order to save the part of the code rate overhead, a special mode for comparatively saving motion information is designed: skip mode and Direct mode.
In Skip mode and Direct mode, the motion information of the current block completely multiplexes the motion information of some neighboring block in the temporal or spatial domain, i.e., one motion information is selected as the motion information of the current block from the motion information sets of a plurality of surrounding blocks.
Therefore, this mode only needs to encode one index value to indicate which motion information in the motion information set is used by the current block, and the difference between Skip mode and Direct mode is that: the Skip mode does not need to encode residual, the Direct mode needs to encode residual, the Direct mode is a Direct mode in interframe prediction and needs to transmit residual information, a decoding end can deduce motion information of a current block by analyzing indexes, and a reconstructed value is obtained by adding a residual value to a predicted value after obtaining the motion information.
Rate-Distortion principle (Rate-Distortion Optimized): there are two major indicators for evaluating coding efficiency: code rate and Peak Signal to Noise Ratio (PSNR), the smaller the bit stream, the larger the compression rate, and the larger the PSNR, the better the reconstructed image quality, and in the mode selection, the discrimination formula is essentially the comprehensive evaluation of the two. For example, the cost for a mode: j (mode) = D + λ R, where D represents Distortion, which can be generally measured using SSE index, where SSE refers to the mean square sum of the differences between the reconstructed image block and the source image; and λ is a lagrange multiplier, and R is the actual number of bits required for encoding the image block in the mode, including the sum of bits required for encoding mode information, motion information, residual errors and the like. When selecting the mode, if the RDO principle is used to make a comparison decision on the coding mode, the best coding performance can be ensured.
In the inter-frame prediction process, a motion information candidate list is usually required to be constructed for the current block, the motion information candidate list may include a plurality of candidate motion information, and the candidate motion information in the motion information candidate list may be spatial motion information or temporal motion information. Obviously, in order to construct the motion information candidate list, the temporal motion information of the current block needs to be acquired, and the temporal motion information acquired in the related art has the problems of poor prediction effect, poor coding performance and the like.
In view of the above findings, in this embodiment, when the forward motion information of the time domain neighboring block is unavailable and the backward motion information of the time domain neighboring block is available, the forward motion information in the time domain motion information of the current block may be determined according to the backward motion information of the time domain neighboring block, and the backward motion information in the time domain motion information of the current block may be determined according to the backward motion information of the time domain neighboring block, so as to obtain the time domain motion information of the current block, improve the accuracy of prediction, improve the prediction performance, and improve the encoding performance.
The following describes a method for constructing a motion information candidate list in detail with reference to several embodiments.
Example 1: referring to fig. 2, a flowchart of a method for constructing a motion information candidate list, which may be applied to a decoding side (also referred to as a video decoder) or an encoding side (also referred to as a video encoder), may include:
in step 201, if the current frame where the current block is located is a B frame, the time domain neighboring block of the current block is determined from the co-located frame of the current frame.
For example, if the current frame where the current block is located is a B frame, a time-domain neighboring block (also referred to as a collocated position) corresponding to a preset position may be determined from a co-located frame (also referred to as a collocated frame) of the current frame based on the preset position of the current block.
For example, the preset position of the current block may be configured according to experience, and the preset position of the current block is not limited, for example, the preset position of the current block may be: the pixel position of the upper left corner of the current block, or the pixel position of the upper right corner of the current block, or the pixel position of the lower left corner of the current block, or the pixel position of the lower right corner of the current block, or the pixel position of the center of the current block, etc.
For example, the collocated frame (collocated frame) may be a corresponding frame of a reference frame index in a backward reference frame list of the current frame, such as a corresponding frame of a reference frame index of 0 in the backward reference frame list, or a corresponding frame of a reference frame index of 1 in the backward reference frame list, or a corresponding frame of a reference frame index of 2 in the backward reference frame list, which is not limited herein. The collocated frame may also be a frame corresponding to a reference frame index in a forward reference frame list of the current frame, for example, a frame corresponding to a reference frame index in a forward reference frame list of 0, or a frame corresponding to a reference frame index in a forward reference frame list of 1, or a frame corresponding to a reference frame index in a forward reference frame list of 2, which is not limited in this regard. Of course, the above are only a few examples and are not limiting.
In a possible implementation manner, the co-located frame is determined in such a way that, if the current frame is a P frame, the co-located frame of the current frame is a frame with a reference frame index of 0 in a forward reference frame List (i.e., reference frame List0, the forward reference frame List may also be referred to as reference picture queue 0), and if the current frame is a B frame, the co-located frame of the current frame is a frame with a reference frame index of 0 in a backward reference frame List (i.e., reference frame List1, the backward reference frame List may also be referred to as reference picture queue 1).
For example, the acquisition manner of the collocated frame (collocated frame) may be a preset collocated frame, such as taking the first reference frame in the forward reference frame list of the current frame as the collocated frame, or taking the first reference frame in the backward reference frame list of the current frame as the collocated frame; or, the derived co-located frame may also be, for example, a reference frame closest to the current frame in the forward reference frame list of the current frame is used as the co-located frame, or a reference frame closest to the current frame in the backward reference frame list of the current frame is used as the co-located frame; or, the reference frame may be a reference frame parsed from the encoded bitstream, for example, for a decoding end, the decoding end may parse the indication information of the co-located frame from the code stream, and determine the co-located frame according to the indication information of the co-located frame.
Illustratively, the collocated position is a time-domain neighboring block in the co-located frame corresponding to the preset position of the current block.
Temporal motion information of the current block, which may include forward motion information and backward motion information, is determined based on motion information of temporal neighboring blocks, step 202. For example, if the forward motion information of the temporal neighboring block is not available and the backward motion information of the temporal neighboring block is available, the backward motion information in the temporal motion information is determined according to the backward motion information of the temporal neighboring block.
In another possible implementation, if the forward motion information of the time-domain neighboring block is available and the backward motion information of the time-domain neighboring block is not available, the forward motion information in the time-domain motion information is determined according to the forward motion information of the time-domain neighboring block, and the backward motion information in the time-domain motion information is determined according to the forward motion information of the time-domain neighboring block.
In another possible implementation, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is available, the forward motion information in the temporal motion information is determined according to the forward motion information of the temporal neighboring block, and the backward motion information in the temporal motion information is determined according to the backward motion information of the temporal neighboring block.
In another possible implementation, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is available, the forward motion information in the temporal motion information is determined according to the forward motion information of the temporal neighboring block, and the backward motion information in the temporal motion information is determined according to the forward motion information of the temporal neighboring block.
In another possible embodiment, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is not available, the forward motion information in the temporal motion information is determined according to the forward motion information of the temporal neighboring block.
In another possible implementation, if the forward motion information of the temporal neighboring block is not available and the backward motion information of the temporal neighboring block is available, the forward motion information in the temporal motion information is determined according to the backward motion information of the temporal neighboring block, and the backward motion information in the temporal motion information is determined according to the backward motion information of the temporal neighboring block.
In the above embodiment, the motion information of the temporal neighboring block comprises forward motion information of the temporal neighboring block and backward motion information of the temporal neighboring block. The forward motion information of the temporal neighboring block includes forward motion vectors of the temporal neighboring block and forward reference frame indices of the temporal neighboring block, which correspond to forward reference frames of the temporal neighboring block. The backward motion information of the temporal neighboring block includes backward motion vectors of the temporal neighboring block and backward reference frame indices of the temporal neighboring block, the backward reference frame indices corresponding to backward reference frames of the temporal neighboring block. The temporal motion information of the current block includes forward motion information and backward motion information, and the forward motion information in the temporal motion information includes a forward motion vector of the current block and a forward reference frame index of the current block, which corresponds to a forward reference frame of the current block. The forward reference frame of the current block is a reference frame in the forward reference frame list of the current block, for example, a reference frame with a reference frame index of 0 in the forward reference frame list of the current block, or a reference frame with a reference frame index of 1 in the forward reference frame list of the current block, which is not limited thereto. The backward motion information in the temporal motion information includes a backward motion vector of the current block and a backward reference frame index of the current block, which corresponds to a backward reference frame of the current block. The backward reference frame of the current block is a reference frame in the backward reference frame list of the current block, for example, a reference frame with a reference frame index of 0 in the backward reference frame list of the current block, or a reference frame with a reference frame index of 1 in the backward reference frame list of the current block, which is not limited thereto.
For example, determining the forward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring block may include, but is not limited to: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the forward motion vectors of the time domain adjacent block to obtain stretched motion vectors, and determining the stretched motion vectors as the forward motion vectors of the current block; determining forward motion information among temporal motion information of the current block based on the forward motion vector of the current block and a forward reference frame index of the current block. For example, the forward motion information in the temporal motion information of the current block includes the forward motion vector of the current block and a forward reference frame index of the current block.
For example, determining the backward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block may include, but is not limited to: based on the distance between the collocated frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, the backward motion vectors of the time domain adjacent blocks are stretched to obtain stretched motion vectors, and the stretched motion vectors are determined as the backward motion vectors of the current block; determining backward motion information among temporal motion information of the current block based on the backward motion vector of the current block and a backward reference frame index of the current block. For example, the backward motion information in the temporal motion information of the current block includes the backward motion vector of the current block and a backward reference frame index of the current block.
For example, determining the forward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block may include, but is not limited to: based on the distance between the co-located frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, performing stretching on the backward motion vectors of the time domain adjacent block to obtain stretched motion vectors, and determining the stretched motion vectors as the forward motion vectors of the current block; determining forward motion information among temporal motion information of the current block based on the forward motion vector of the current block and a forward reference frame index of the current block. For example, the forward motion information in the temporal motion information of the current block includes the forward motion vector of the current block and a forward reference frame index of the current block.
For example, determining backward motion information in the temporal motion information of the current block according to forward motion information of the temporal neighboring block may include, but is not limited to: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information among temporal motion information of the current block based on the backward motion vector of the current block and a backward reference frame index of the current block. For example, the backward motion information in the temporal motion information of the current block includes the backward motion vector of the current block and a backward reference frame index of the current block.
Step 203, constructing a motion information candidate list of the current block based on the temporal motion information of the current block.
According to the technical scheme, in the embodiment of the application, when the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, the forward motion information in the time domain motion information of the current block can be determined according to the backward motion information of the time domain adjacent block, and the backward motion information in the time domain motion information of the current block is determined according to the backward motion information of the time domain adjacent block, so that the time domain motion information of the current block is obtained, the accuracy of prediction is improved, the prediction performance is improved, the coding performance is improved, the predicted value of the current block can be closer to the original pixel, and the coding performance is improved.
Example 2: in order to obtain the temporal motion information of the current block, the following method may be used:
if the current frame of the current block is a P frame, namely the current image is a P image, the following steps are adopted to determine the time domain motion information:
step a1, based on the preset position of the current block, determining a time domain adjacent block corresponding to the preset position from a co-located frame of the current frame, wherein if the motion information of the time domain adjacent block is unavailable, the time domain motion information of the current block is zero motion information.
Illustratively, the co-located frame may be a picture with reference index 0 in a forward reference frame List of the current frame, which may also be referred to as reference picture queue 0, i.e., list0. The preset position of the current block may be an upper left corner luma sample position. If the reference frame index of the motion information of the temporal neighboring block is-1, it indicates that the motion information of the temporal neighboring block is not available.
In summary, step a1 can also be expressed as: if the reference frame index stored in the time domain motion information storage unit where the brightness sample (i.e., the time domain adjacent block) corresponding to the brightness sample position at the top left corner of the current block in the image with the reference index of 0 in the reference image queue 0 is located is-1 (indicating that the motion information is unavailable), the L0 motion vector MvE0 of the current block is a zero vector, and the L0 reference index value RefIdxL0 of the current block is 0, ending the time domain motion information derivation process.
Step a2, if the motion information of the time domain adjacent block is available, the L0 reference index RefIdxL0 of the current block is 0, the distance index of the image corresponding to the L0 reference index is denoted as DistanceIndexL0, and the block distance of the image corresponding to the L0 reference index is denoted as BlockDistanceL0. The motion vector of the time domain neighboring block is denoted as mvRef (mvRef _ x, mvRef _ y), the distance index of the image where the time domain neighboring block is located is denoted as DistanceIndexCol, and the distance index where the motion vector points to the image is denoted as distanceindexcef.
For example, if the reference frame index stored in the temporal motion information storage unit where the luma sample (i.e., the temporal neighboring block) corresponding to the position of the luma sample at the top-left corner of the current block in the image with the reference index of 0 in the reference image queue 0 is located is not-1, if the stored reference frame index is greater than or equal to 0, it indicates that the motion information of the temporal neighboring block is available.
Illustratively, the L0 reference index RefIdxL0 of the current block is 0, which indicates the first picture in the reference picture queue 0, i.e., the L0 reference index corresponds to the first picture in the reference picture queue 0. The distance index distanceindex L0 of the picture corresponding to the L0 reference index indicates the distance index of the first picture in the reference picture queue 0, such as POC (display order). The block distance blockdisttancel 0 of the picture corresponding to the L0 reference index represents the distance between the picture corresponding to the L0 reference index and the current frame, for example, the difference between the distance index DistanceIndexL0 of the first picture in the reference picture queue 0 and the distance index of the current frame.
Illustratively, in the reference picture queue 0, the L0 motion vector of the temporal motion information storage unit in which the luminance sample corresponding to the top-left luminance sample position of the current block in the picture with reference index 0 is located is denoted as mvRef (mvRef _ x, mvRef _ y), i.e., a motion vector of the temporal neighboring block. The distance index of the image where the motion information storage unit is located is denoted as DistanceIndexCol, and the distance index of the image where the reference unit pointed to by the motion vector is denoted as distanceindexcef.
Step a3, based on the DistanceIndexCol and distanceindexcef, the distance BlockDistanceRef between the image in which the temporal neighboring block is located and the image to which the motion vector of the temporal neighboring block points may be determined in the following manner.
BlockDistanceRef＝DistanceIndexCol–DistanceIndexRef
Step a4, let the L0 reference index RefIdxL0 of the current block be 0, determine the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) of the current block by using the following formula, and the L0 reference index and the L0 motion vector constitute the temporal motion information of the current block.
mvE0_x＝Clip3(-32768,32767,Sign(mvRef_x*BlockDistanceL0*BlockDistanceRef)*(((Abs(mvRef_x*BlockDistanceL0*(16384/BlockDistanceRef)))+8192)>>14))；
mvE0_y＝Clip3(-32768,32767,Sign(mvRef_y*BlockDistanceL0*BlockDistanceRef)*(((Abs(mvRef_y*BlockDistanceL0*(16384/BlockDistanceRef)))+8192)>>14))。
In the above formula, clip3 (-32768, 32767) is used to limit the value ranges of mvE0_ x and mvE0_ y between-32768 and 32767, that is, when the value is less than-32768, the value of mvE0_ x and mvE0_ y is-32768, and when the value is greater than 32767, the value of mvE0_ x and mvE0_ y is 32767.Sign denotes Sign function, abs denotes absolute value, and > > denotes right shift. mvRef _ x represents a horizontal motion vector of a time domain neighboring block, and mvRef _ y represents a vertical motion vector of the time domain neighboring block. mvE0_ x denotes a horizontal motion vector of temporal motion information of the current block, and mvE0_ y denotes a vertical motion vector of temporal motion information of the current block. For the definition of BlockDistanceL0 and blockdistancefef, see the above steps, which are not described herein. Of course, each numerical value in the above formula can be adjusted according to actual needs, and is not limited.
Step a5, let the value of interPredRefMode equal to 'PRED _ List0'.
Based on the above steps, if the current frame is a P frame, the temporal motion information of the current block can be obtained.
If the current frame where the current block is located is a B frame, namely the current image is a B image, the time domain motion information is determined by adopting the following steps:
b1, based on the preset position of the current block, determining a time domain adjacent block corresponding to the preset position from a same-position frame of the current frame, and if the forward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to an MVP mode.
For example, the co-located frame may be a picture with a reference index of 0 in a backward reference frame List of the current frame, and the backward reference frame List may also be referred to as a reference picture queue 1, i.e., list1. The preset position of the current block may be an upper left corner luma sample position. If the reference frame index of the motion information of the temporal neighboring block is-1, it indicates that the forward motion information of the temporal neighboring block is not available.
For example, obtaining the temporal motion information of the current block according to the MVP method may include, but is not limited to: and taking the size and the position of the coding unit where the current block is located as the size and the position of the current block, respectively taking the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to MVP as the L0 motion vector MvE0 and the L1 motion vector MvE1 of the current block, making the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block equal to 0, and ending the motion information derivation process.
In summary, step b1 can also be expressed as: if the reference frame index stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is-1 (the storage unit stores the reference frame index of the forward motion information, and the reference frame index of-1 indicates that the forward motion information is unavailable), then both the L0 reference index and the L1 reference index of the current block are equal to 0. And taking the size and the position of the coding unit where the current block is located as the size and the position of the current block, respectively taking the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to MVP as the L0 motion vector MvE0 and the L1 motion vector MvE1 of the current block, making the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block equal to 0, and ending the time domain motion information derivation process of the current block.
Step b2, if the forward motion information of the temporal neighboring block is available, the L0 reference index and the L1 reference index of the current block may both be equal to 0. The distance index of the image corresponding to the L0 reference index is denoted as distanceindexexl 0, and the distance index of the image corresponding to the L1 reference index is denoted as distanceindexexl 1; the block distance of the image corresponding to the L0 reference index is denoted as blockDistanceL0, and the block distance of the image corresponding to the L1 reference index is denoted as blockDistanceL1.
For example, if the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located stores a reference frame index that is not-1, e.g., the reference frame index is greater than or equal to 0, then the forward motion information of the temporal neighboring block is available.
Illustratively, the L0 reference index of the current block is 0, which indicates the first reference picture in the reference picture queue 0 of the current block, i.e., the L0 reference index corresponds to the first picture in the reference picture queue 0. The distance index distanceindex L0 of the picture corresponding to the L0 reference index represents the distance index of the first picture in the reference picture queue 0, such as POC of the first picture. The block distance blockdistance L0 of the picture corresponding to the L0 reference index represents the distance between the picture corresponding to the L0 reference index and the current frame, for example, the difference between the distance index of the first picture in the reference picture queue 0 and the distance index of the current frame.
Illustratively, the L1 reference index of the current block is 0, which indicates the first picture in the reference picture queue 1 of the current block, i.e., the L1 reference index corresponds to the first picture in the reference picture queue 1. The distance index distanceindex L1 of the picture corresponding to the L1 reference index represents the distance index of the first picture in the reference picture queue 1, such as POC of the first reference picture. The block distance blockdistance L1 of the picture corresponding to the L1 reference index represents a distance between the picture corresponding to the L1 reference index and the current frame, for example, a difference between the distance index of the first picture in the reference picture queue 1 and the distance index of the current frame.
And step b3, recording the motion vector of the time domain adjacent block as mvRef (mvRef _ x, mvRef _ y), recording the distance index of the image in which the time domain adjacent block is located as DistanceIndexCol, and recording the distance index of the reference image pointed by the motion vector of the time domain adjacent block as distanceindexcef. For example, in the reference picture queue 1, the L0 motion vector of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the picture with reference index 0 is located is denoted as mvRef (mvRef _ x, mvRef _ y), i.e., the motion vector of the temporal neighboring block. The distance index of the image in which the motion information storage unit is located is denoted as DistanceIndexCol, and the distance index of the image in which the reference unit to which the motion vector points is located is denoted as distanceindexcef. Based on the DistanceIndexCol and DistanceIndexRef, the distance BlockDistanceRef between the image in which the temporal neighboring block is located and the reference image to which the motion vector of the temporal neighboring block points can be determined as follows.
BlockDistanceRef＝DistanceIndexCol–DistanceIndexRef
And step b4, setting the L0 reference index RefIdxL0 of the current block to be 0, and determining the L0 motion vector mvE0 (mvE 0_ x and mvE0_ y) of the current block by adopting the following formula, wherein the L0 reference index and the L0 motion vector form the forward motion information in the time domain motion information of the current block. Let L1 reference index RefIdxL1 of the current block be 0, determine L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) of the current block by using the following formula, where the L1 reference index and L1 motion vector constitute backward motion information in the temporal motion information of the current block, and the forward motion information and the backward motion information constitute temporal motion information of the current block.
mvE0_x＝Clip3(-32768,32767,Sign(mvRef_x*BlockDistanceL0*BlockDistanceRef)*(((Abs(mvRef_x*BlockDistanceL0*(16384/BlockDistanceRef)))+8192)>>14))
mvE0_y＝Clip3(-32768,32767,Sign(mvRef_y*BlockDistanceL0*BlockDistanceRef)*(((Abs(mvRef_y*BlockDistanceL0*(16384/BlockDistanceRef)))+8192)>>14))
mvE1_x＝Clip3(-32768,32767,Sign(mvRef_x*BlockDistanceL1*BlockDistanceRef)*(((Abs(mvRef_x*BlockDistanceL1*(16384/BlockDistanceRef)))+8192)>>14))
mvE1_y＝Clip3(-32768,32767,Sign(mvRef_y*BlockDistanceL1*BlockDistanceRef)*(((Abs(mvRef_y*BlockDistanceL1*(16384/BlockDistanceRef)))+8192)>>14))
In the above formula, the meaning of each symbol can be referred to as step a4, and is not repeated here.
Based on the above steps, if the current frame is a B frame, the temporal motion information of the current block can be obtained.
In embodiment 2, if the current frame is a B frame, the L0 motion information of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with reference index 0 in the reference image queue 1 is located is used as the reference motion information, and the L1 motion information of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with reference index 0 in the reference image queue 1 is not considered as the reference motion information.
Example 3: in order to obtain the temporal motion information of the current block, if the current frame where the current block is located is a P frame, that is, the current picture is a P picture, the determination method of the temporal motion information is described in embodiment 2, and is not described herein again. If the current frame where the current block is located is a B frame, namely the current image is a B image, the following steps are adopted to determine the time domain motion information of the current block. Illustratively, a P frame is an image that can be decoded in inter prediction using a knowledge image and a past image in display order as reference images. B frames are pictures that can be decoded using a knowledge picture in inter prediction, and display the past and future pictures in order as reference pictures.
Step c1, based on the preset position of the current block, determining a time domain adjacent block corresponding to the preset position from a co-located frame of the current frame, wherein the motion information of the time domain adjacent block comprises forward motion information and backward motion information. And if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to the MVP mode.
For example, the collocated frame of the current frame (collocated frame) may be a corresponding image (image may also be referred to as a frame) with a reference frame index in a backward reference frame list of the current frame, such as a corresponding image with a reference frame index of 0 in the backward reference frame list, or may be a corresponding image with a reference frame index in a forward reference frame list of the current frame, such as a corresponding image with a reference frame index of 0 in the forward reference frame list, and the collocated frame is not limited. For convenience of description, taking an image with a reference index of 0 in the backward reference frame List of the current frame as an example, the backward reference frame List may also be referred to as a reference image queue 1, i.e., list1.
For example, the preset position of the current block may be configured according to experience, for example, the upper corner luma sample position, the upper right luma sample position, the lower left luma sample position, the lower right luma sample position, the center luma sample position, and the like, and the preset position is not limited. For convenience of description, the top-left corner luma sample position of the current block is taken as an example.
Illustratively, if the reference frame index of the forward motion information of the temporal neighboring block is-1, it indicates that the forward motion information of the temporal neighboring block is not available. If the reference frame index of the forward motion information of the temporal neighboring block is not-1, e.g., the reference frame index is greater than or equal to 0, it indicates that the forward motion information of the temporal neighboring block is available. If the reference frame index of the backward motion information of the temporal neighboring block is-1, it indicates that the backward motion information of the temporal neighboring block is not available. If the reference frame index of the backward motion information of the temporal neighboring block is not-1, if the reference frame index is greater than or equal to 0, it indicates that the backward motion information of the temporal neighboring block is available.
For example, obtaining the temporal motion information of the current block according to the MVP method may include, but is not limited to, the following methods: the size and the position of the coding unit where the current block is located are used as the size and the position of the current block, the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to the MVP are respectively used as an L0 motion vector MvE0 and an L1 motion vector MvE1 of the current block, and an L0 reference index RefIdxL0 and an L1 reference index RefIdxL1 of the current block are both equal to 0.
The L0 motion vector MvE0 and the L0 reference index RefIdxL0 constitute forward motion information in the temporal motion information of the current block, and the L1 motion vector MvE1 and the L1 reference index RefIdxL1 constitute backward motion information in the temporal motion information of the current block, and the forward motion information and the backward motion information constitute temporal motion information of the current block.
The L0 motion vector MvE0 represents a forward motion vector, the L0 reference index RefIdxL0 represents a forward reference index, the L1 motion vector MvE1 represents a backward motion vector, and the L1 reference index RefIdxL1 represents a backward reference index.
And c2, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block.
Illustratively, if the reference frame index of the forward motion information of the temporal neighboring block is not-1, e.g., the reference frame index is greater than or equal to 0, then the forward motion information of the temporal neighboring block is available. If the reference frame index of the backward motion information of the temporal neighboring block is not-1, if the reference frame index is greater than or equal to 0, it indicates that the backward motion information of the temporal neighboring block is available.
The forward motion information of the time domain neighboring block may include a forward motion vector mvRef0 (mvRef 0_ x, mvRef0_ y) of the time domain neighboring block and a forward reference frame index of the time domain neighboring block, the forward reference frame index corresponding to a forward reference frame of the time domain neighboring block, and the backward motion information of the time domain neighboring block may include a backward motion vector mvRef1 (mvRef 1_ x, mvRef1_ y) of the time domain neighboring block and a backward reference frame index of the time domain neighboring block, the backward reference frame index corresponding to a backward reference frame of the time domain neighboring block.
The forward motion information in the temporal motion information of the current block includes a forward motion vector mvE0 (mvE 0_ x, mvE0_ y) of the current block and a forward reference frame index of the current block, where the forward reference frame index corresponds to a forward reference frame of the current block, the forward reference frame of the current block is a reference frame in a forward reference frame list of the current block, and taking a reference frame whose reference frame index in the forward reference frame list of the current block is 0 as an example, that is, a reference frame index (may also be referred to as L0 reference index) of the forward reference frame of the current block is 0, which represents a first image in the forward reference frame list (i.e., reference image queue 0) of the current block. The backward motion information in the temporal motion information of the current block includes a backward motion vector mvE1 (mvE 1_ x, mvE1_ y) of the current block and a backward reference frame index of the current block, the backward reference frame index corresponds to a backward reference frame of the current block, the backward reference frame of the current block is a reference frame in a backward reference frame list of the current block, taking a reference frame of which the reference frame index in the backward reference frame list of the current block is 0 as an example, that is, the reference frame index (which may also be referred to as L1 reference index) of the backward reference frame of the current block is 0, which represents the first image in the backward reference frame list (i.e., reference image queue 1) of the current block.
For example, a distance index (e.g., POC) of a forward reference frame of the current block may be denoted as distanceindex l0, a distance index of a backward reference frame of the current block may be denoted as distanceindex l1, a distance between the current frame and the forward reference frame of the current block may be denoted as blockadiscancel 0, the blockadiscancel 0 may represent a difference between the distance index of the current frame and the distance index of the forward reference frame of the current block, and a distance between the current frame and the backward reference frame of the current block may be denoted as blockadiscancel 1, and the blockadiscancel 1 may represent a difference between the distance index of the current frame and the distance index of the backward reference frame of the current block.
For example, a distance index of an image in which the temporal neighboring block is located (i.e., a co-located frame of the current frame) may be denoted as DistanceIndexCol, a distance index of an image to which the forward motion vector mvRef0 of the temporal neighboring block points (i.e., a forward reference frame of the temporal neighboring block) may be denoted as distanceindexcef 0, and a distance index of an image to which the backward motion vector mvRef1 of the temporal neighboring block points (i.e., a backward reference frame of the temporal neighboring block) may be denoted distanceindexcef 1. Based on this, the distance between the co-located frame and the forward reference frame of the temporal neighboring block can be denoted as blockdistancefef 0, and the distance between the co-located frame and the backward reference frame of the temporal neighboring block can be denoted as blockdistancefef 1.
The blockdistancefef 0 represents a difference value between the distance index of the co-located frame and the distance index of the forward reference frame of the temporal neighboring block, and may be represented by the following formula, for example: blockDistanceRef0= DistanceIndexCol-DistanceIndexRef0.
The blockdistanceff 1 represents a difference value between a distance index of a co-located frame and a distance index of a backward reference frame of a temporal neighboring block, and may be represented by, for example, the following formula: blockDistanceRef1= DistanceIndexCol-DistanceIndexRef1.
Based on the above definition, determining forward motion information in the temporal motion information of the current block according to forward motion information of the temporal neighboring block may include, but is not limited to: based on the distance blockdistancefef 0 between the co-located frame and the forward reference frame of the time domain adjacent block and the distance BlockDistanceL0 between the current frame and the forward reference frame of the current block, performing stretching on a forward motion vector mvRef0 (mvRef 0_ x, mvRef0_ y) of the time domain adjacent block to obtain a stretched motion vector, and taking the stretched motion vector as a forward motion vector mvE0 (mvE 0_ x, mvE0_ y) of the current block; forward motion information in temporal motion information of the current block is determined based on the forward motion vector of the current block and a forward reference frame index of the current block, and for example, the forward motion information in the temporal motion information includes mvE0 (mvE 0_ x, mvE0_ y) and the forward reference frame index of the current block.
For example, the following equations (1) and (2) may be used to scale forward motion vectors mvRef0 (mvRef 0_ x, mvRef0_ y) of the time-domain neighboring blocks, so as to obtain scaled motion vectors mvE0 (mvE 0_ x, mvE0_ y).
mvE0_x＝Clip3(-32768,32767,Sign(mvRef0_x*BlockDistanceL0*BlockDistanceRef0)*(((Abs(mvRef0_x*BlockDistanceL0*(16384/BlockDistanceRef0)))+8192)>>14))(1)
mvE0_y＝Clip3(-32768,32767,Sign(mvRef0_y*BlockDistanceL0*BlockDistanceRef0)*(((Abs(mvRef0_y*BlockDistanceL0*(16384/BlockDistanceRef0)))+8192)>>14))(2)
In the formula (1) and the formula (2), clip3 (-32768, 32767) is used to limit the value ranges of mvE0_ x and mvE0_ y between-32768 and 32767, that is, when the value is less than-32768, the value of mvE0_ x and mvE0_ y is-32768, and when the value is more than 32767, the value of mvE0_ x and mvE0_ y is 32767.Sign denotes Sign function, abs denotes absolute value, and > > denotes right shift. mvRef0_ x represents a horizontal motion vector of a forward motion vector of a temporal neighboring block, and mvRef0_ y represents a vertical motion vector of a forward motion vector of a temporal neighboring block. mvE0_ x denotes a horizontal motion vector of a forward motion vector of the current block, and mvE0_ y denotes a vertical motion vector of the forward motion vector of the current block. The BlockDistanceL0 represents a distance between the current frame and a forward reference frame of the current block, and the blockdistancefef 0 represents a distance between the co-located frame and a forward reference frame of the temporal neighboring block. The numerical values in the formula (1) and the formula (2) can be adjusted according to actual needs, and are not limited. Of course, the formula (1) and the formula (2) are only examples, and it is only necessary to stretch the forward motion vector mvRef0 of the time domain neighboring block based on the distance between the co-located frame and the forward reference frame of the time domain neighboring block and the distance between the current frame and the forward reference frame of the current block to obtain a stretched motion vector mvE0. For example, based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, the horizontal motion vector in the forward motion vector of the time domain adjacent block is extended to obtain the horizontal motion vector in the forward motion vector of the current block; and based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the vertical motion vector in the forward motion vector of the time domain adjacent block to obtain the vertical motion vector in the forward motion vector of the current block.
Based on the above definition, determining backward motion information in temporal motion information of the current block according to backward motion information of the temporal neighboring block includes, but is not limited to: based on a distance blockdistancefef 1 between a collocated frame and a backward reference frame of a time-domain adjacent block and a distance BlockDistanceL1 between a current frame and a backward reference frame of a current block, performing stretching on a backward motion vector mvRef1 (mvRef 1_ x, mvRef1_ y) of the time-domain adjacent block to obtain a stretched motion vector, and taking the stretched motion vector as a backward motion vector mvE1 (mvE 1_ x, mvE1_ y) of the current block; the backward motion information in the temporal motion information of the current block is determined based on the backward motion vector of the current block and the backward reference frame index of the current block, and for example, the backward motion information in the temporal motion information of the current block includes mvE1 (mvE 1_ x, mvE1_ y) and the backward reference frame index of the current block.
For example, the following formulas (3) and (4) may be used to stretch the backward motion vector mvRef1 (mvRef 1_ x, mvRef1_ y) of the time-domain neighboring block to obtain a stretched motion vector mvE1 (mvE 1_ x, mvE1_ y).
mvE1_x＝Clip3(-32768,32767,Sign(mvRef1_x*BlockDistanceL1*BlockDistanceRef1)*(((Abs(mvRef1_x*BlockDistanceL1*(16384/BlockDistanceRef1)))+8192)>>14))(3)
mvE1_y＝Clip3(-32768,32767,Sign(mvRef1_y*BlockDistanceL1*BlockDistanceRef1)*(((Abs(mvRef1_y*BlockDistanceL1*(16384/BlockDistanceRef1)))+8192)>>14))(4)
In the formula (3) and the formula (4), clip3 (-32768, 32767) is used to limit the value ranges of mvE1_ x and mvE1_ y between-32768 and 32767, that is, when the value is less than-32768, the value of mvE1_ x and mvE1_ y is-32768, and when the value is more than 32767, the value of mvE1_ x and mvE1_ y is 32767.Sign denotes Sign function, abs denotes absolute value, and > > denotes right shift. mvRef1_ x represents a horizontal motion vector of a backward motion vector of a temporal neighboring block, and mvRef1_ y represents a vertical motion vector of a backward motion vector of a temporal neighboring block. mvE1_ x denotes a horizontal motion vector of a backward motion vector of the current block, and mvE1_ y denotes a vertical motion vector of the backward motion vector of the current block. BlockDistanceL1 represents a distance between the current frame and the backward reference frame of the current block, and blockdistancefef 1 represents a distance between the co-located frame and the backward reference frame of the temporal neighboring block. Each numerical value in the formula (3) and the formula (4) can be adjusted according to actual needs, and the adjustment is not limited. Of course, the formula (3) and the formula (4) are only examples, and it is only necessary to stretch the first rear motion vector mvRef1 based on the distance between the collocated frame and the backward reference frame of the time-domain neighboring block and the distance between the current frame and the backward reference frame of the current block, so as to obtain the stretched backward motion vector mvE1 of the current block. For example, based on the distance between the co-located frame and the backward reference frame of the time-domain neighboring block and the distance between the current frame and the backward reference frame of the current block, the horizontal motion vector in the backward motion vectors of the time-domain neighboring block is extended to obtain the horizontal motion vector in the backward motion vector of the current block; and based on the distance between the co-located frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the vertical motion vector in the backward motion vector of the time domain adjacent block to obtain the vertical motion vector in the backward motion vector of the current block.
In summary, the forward motion information in the temporal motion information of the current block and the backward motion information in the temporal motion information of the current block may be obtained, and the forward motion information and the backward motion information may form the temporal motion information of the current block.
In summary, as shown in fig. 3A, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is available, the distance BlockDistanceRef0 between the collocated frame and the forward reference frame of the temporal neighboring block may be determined based on the distance index distancindexcol of the collocated frame and the distance index distancindexcef 0 of the forward reference frame of the temporal neighboring block, the distance BlockDistanceL0 between the current frame and the forward reference frame of the current block may be determined based on the distance index distancindexcur of the current frame and the distance index distancindexcl 0 of the forward reference frame of the current block, and then the forward motion vector mv ref0 of the temporal neighboring block is extended through the blockdistanciref 0 and the forward motion vector mvRef0 to obtain the forward motion vector mvE0 of the current block. The distance BlockDistanceRef1 between the collocated frame and the backward reference frame of the temporal neighboring block may be determined based on the distance index distancindexcol of the collocated frame and the distance index distancindexref 1 of the backward reference frame of the temporal neighboring block, the distance blockdistancerel 1 between the current frame and the backward reference frame of the current block may be determined based on the distance index distancindexcur of the current frame and the distance index distancindexcl 1 of the backward reference frame of the current block, and then the backward motion vector mvE1 of the current block is obtained by expanding and contracting the BlockDistanceRef1 and the backward motion vector mvRef1 of the temporal neighboring block.
Then, the forward motion vector mvE0 of the current block, the backward motion vector mvE1 of the current block, the forward reference frame index of the current block and the backward reference frame index of the current block can constitute the temporal motion information of the current block.
And c3, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is unavailable, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block.
If the reference frame index of the forward motion information of the temporal neighboring block is greater than or equal to 0, the forward motion information of the temporal neighboring block is available. If the reference frame index of the backward motion information of the temporal neighboring block is-1, the backward motion information of the temporal neighboring block is unavailable.
The forward motion information of the temporal neighboring block includes a forward motion vector mvRef0 (mvRef 0_ x, mvRef0_ y) of the temporal neighboring block and a forward reference frame index of the temporal neighboring block, the forward reference frame index corresponding to a forward reference frame of the temporal neighboring block, and the backward motion information of the temporal neighboring block includes a backward motion vector mvRef1 (mvRef 1_ x, mv ref1_ y) of the temporal neighboring block and a backward reference frame index of the temporal neighboring block, the backward reference frame index corresponding to a backward reference frame of the temporal neighboring block. The forward motion information in the temporal motion information of the current block includes a forward motion vector mvE0 (mvE 0_ x, mvE0_ y) of the current block and a forward reference frame index of the current block, the forward reference frame index corresponding to a forward reference frame of the current block, and the backward motion information in the temporal motion information of the current block includes a backward motion vector mvE1 (mvE 1_ x, mvE1_ y) of the current block and a backward reference frame index of the current block, the backward reference frame index corresponding to a backward reference frame of the current block. The distance index of the forward reference frame of the current block is recorded as DistanceIndexL0, the distance index of the backward reference frame of the current block is recorded as DistanceIndexL1, the distance between the current frame and the forward reference frame of the current block is recorded as BlockDistanceL0, and the distance between the current frame and the backward reference frame of the current block is recorded as BlockDistanceL1. The distance index of the co-located frame is denoted as DistanceIndexCol, the distance index of the forward reference frame of the time domain neighboring block is denoted as distanceindexerref 0, and the distance index of the backward reference frame of the time domain neighboring block is denoted as distanceindexerref 1. The distance between the co-located frame and the forward reference frame of the time domain neighboring block is denoted as blockdistancefef 0, and the distance between the co-located frame and the backward reference frame of the time domain neighboring block is denoted as blockdistancefef 1.
Based on the above definition, the forward motion information in the temporal motion information of the current block may be determined according to the forward motion information of the temporal neighboring blocks, and the process may refer to the processing process in step c2, which is not repeated herein.
Based on the above definition, determining backward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring block may include, but is not limited to: based on a distance blockdistancefef 0 between a co-located frame and a forward reference frame of a time domain adjacent block and a distance BlockDistanceL1 between a current frame and a backward reference frame of a current block, performing stretching on a forward motion vector mvRef0 (mvRef 0_ x, mvRef0_ y) of the time domain adjacent block to obtain a stretched motion vector, and taking the stretched motion vector as a backward motion vector mvE1 (mvE 1_ x, mvE1_ y) of the current block; the backward motion information in the temporal motion information of the current block is determined based on the backward motion vector of the current block and the backward reference frame index of the current block, and for example, the backward motion information in the temporal motion information includes mvE1 (mvE 1_ x, mvE1_ y) and the backward reference frame index of the current block.
For example, the following equations (5) and (6) may be used to scale forward motion vectors mvRef0 (mvRef 0_ x, mvRef0_ y) of the time-domain neighboring blocks to obtain scaled motion vectors mvE1 (mvE 1_ x, mvE1_ y).
mvE1_x＝Clip3(-32768,32767,Sign(mvRef0_x*BlockDistanceL1*BlockDistanceRef0)*(((Abs(mvRef0_x*BlockDistanceL1*(16384/BlockDistanceRef0)))+8192)>>14))(5)
mvE1_y＝Clip3(-32768,32767,Sign(mvRef0_y*BlockDistanceL1*BlockDistanceRef0)*(((Abs(mvRef0_y*BlockDistanceL1*(16384/BlockDistanceRef0)))+8192)>>14))(6)
In equations (5) and (6), clip3 (-32768, 32767) is used to limit the value range of mvE1_ x and mvE1_ y to between-32768 and 32767.Sign denotes Sign function, abs denotes absolute value, and > > denotes right shift. mvRef0_ x represents the horizontal motion vector of the forward motion vector of the temporal neighboring block, and mvRef0_ y represents the vertical motion vector of the forward motion vector of the temporal neighboring block. mvE1_ x denotes a horizontal motion vector of a backward motion vector of the current block, and mvE1_ y denotes a vertical motion vector of the backward motion vector of the current block. BlockDistanceL1 represents the distance between the current frame and the backward reference frame of the current block, and blockdistancefef 0 represents the distance between the co-located frame and the forward reference frame of the temporal neighboring block.
In summary, the forward motion information in the temporal motion information of the current block and the backward motion information in the temporal motion information of the current block may be obtained, and the forward motion information and the backward motion information may form the temporal motion information of the current block.
In summary, as shown in fig. 3B, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is not available, a distance blockandexcel 0 between the co-located frame and the forward reference frame of the temporal neighboring block is determined based on the distance index distancelndexcol of the co-located frame and the distance index distancelndexref 0 of the forward reference frame of the temporal neighboring block, a distance blockandexcir 0 between the current frame and the forward reference frame of the current block is determined based on the distance index distancelndexcurr of the current frame and the distance index distancelndex 0 of the forward reference frame of the current block, and then the forward motion vector mvRef0 of the temporal neighboring block is extended and contracted through the blockadiscutanexeref 0 and the blockandexcel 0 to obtain the forward motion vector mvE0 of the current block. The distance blockdistensance l1 between the current frame and the backward reference frame of the current block may be determined based on the distance index distanceindexccur of the current frame and the distance index distanceindexcl 1 of the backward reference frame of the current block, and the forward motion vector mvRef0 of the temporal neighboring block is extended and contracted through blockdistensaref 0 and blockdistensance l1 to obtain the backward motion vector mvE1 of the current block.
The forward motion vector mvE0 of the current block, the backward motion vector mvE1 of the current block, the forward reference frame index of the current block and the backward reference frame index of the current block can form the time domain motion information of the current block.
And c4, if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block.
If the reference frame index of the forward motion information of the temporal neighboring block is-1, the forward motion information of the temporal neighboring block is unavailable. If the reference frame index of the backward motion information of the temporal neighboring block is greater than or equal to 0, the backward motion information of the temporal neighboring block is available.
Based on the definition of the above embodiment, the backward motion information in the temporal motion information of the current block may be determined according to the backward motion information of the temporal neighboring block, and the process may refer to the processing process in step c2, and is not repeated herein.
Based on the definition of the above embodiment, determining the forward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block may include, but is not limited to: based on the distance blockdistancefef 1 between the co-located frame and the backward reference frame of the time domain adjacent block and the distance BlockDistanceL0 between the current frame and the forward reference frame of the current block, the backward motion vector mvRef1 (mvRef 1_ x, mvRef1_ y) of the time domain adjacent block is stretched to obtain a stretched motion vector, and the stretched motion vector is used as the forward motion vector mvE0 (mvE 0_ x, mvE0_ y) of the current block; determining forward motion information in temporal motion information of the current block based on the forward motion vector of the current block and a forward reference frame index of the current block, for example, the forward motion information in the temporal motion information includes mvE0 (mvE 0_ x, mvE0_ y) and the forward reference frame index of the current block.
For example, the following formulas (7) and (8) may be used to scale the backward motion vector mvRef1 (mvRef 1_ x, mvRef1_ y) of the time-domain neighboring block, so as to obtain a scaled motion vector mvE0 (mvE 0_ x, mvE0_ y).
mvE0_x＝Clip3(-32768,32767,Sign(mvRef1_x*BlockDistanceL0*BlockDistanceRef1)*(((Abs(mvRef1_x*BlockDistanceL0*(16384/BlockDistanceRef1)))+8192)>>14))(7)
mvE0_y＝Clip3(-32768,32767,Sign(mvRef1_y*BlockDistanceL0*BlockDistanceRef1)*(((Abs(mvRef1_y*BlockDistanceL0*(16384/BlockDistanceRef1)))+8192)>>14))(8)
In equations (7) and (8), clip3 (-32768, 32767) is used to limit the value range of mvE0_ x and mvE0_ y to between-32768 and 32767.Sign denotes Sign function, abs denotes absolute value, and > > denotes right shift. mvRef1_ x represents a horizontal motion vector of a backward motion vector of a temporal neighboring block, and mvRef1_ y represents a vertical motion vector of a backward motion vector of a temporal neighboring block. mvE0_ x denotes a horizontal motion vector of a forward motion vector of the current block, and mvE0_ y denotes a vertical motion vector of the forward motion vector of the current block. A blockadistancel 0 represents a distance between the current frame and a forward reference frame of the current block, and a blockadistancef 1 represents a distance between the co-located frame and a backward reference frame of the temporal neighboring block.
In summary, the forward motion information in the temporal motion information of the current block and the backward motion information in the temporal motion information of the current block may be obtained, and the forward motion information and the backward motion information may be combined into the temporal motion information of the current block.
In summary, as shown in fig. 3C, if the forward motion information of the temporal neighboring block is not available and the backward motion information of the temporal neighboring block is available, the distance blockandexcel 1 between the collocated frame and the backward reference frame of the temporal neighboring block is determined based on the distance index distanceindexcel of the collocated frame and the distance index distanceindexcef 1 of the backward reference frame of the temporal neighboring block, the distance blockandexcel 0 between the current frame and the forward reference frame of the current block is determined based on the distance index distancelndex cur of the current frame and the distance index distancelndex 0 of the forward reference frame of the current block, and then the backward motion vector mv ref1 of the temporal neighboring block is extended and contracted through the blockadiscutanef 1 and blockandexcel 0 to obtain the forward motion vector mvE0 of the current block. The distance blockdistensance l1 between the current frame and the backward reference frame of the current block may be determined based on the distance index distanceindexccur of the current frame and the distance index distanceindexcl 1 of the backward reference frame of the current block, and the backward motion vector mvRef1 of the temporal neighboring block is extended and contracted through the blockdistensaref 1 and the blockdistensance l1 to obtain the backward motion vector mvE1 of the current block.
The forward motion vector mvE0 of the current block, the backward motion vector mvE1 of the current block, the forward reference frame index of the current block and the backward reference frame index of the current block can form the time-domain motion information of the current block.
The above process of example 3 is described below with reference to a specific application scenario:
if the L0 reference index and the L1 reference frame index stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located are both-1, then both the L0 reference index and the L1 reference index of the current block are equal to 0. And taking the size and the position of the coding unit where the current block is located as the size and the position of the current block, respectively taking the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to the MVP as an L0 motion vector MvE0 and an L1 motion vector MvE1 of the current block, respectively enabling the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block to be equal to 0, and ending the derivation process of the time domain motion information of the current block.
And if the L0 reference index and the L1 reference frame index stored in the time domain motion information storage unit of the brightness sample corresponding to the upper-left brightness sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 are not both-1, the L0 reference index and the L1 reference index of the current block are both equal to 0. The distance index of the image corresponding to the L0 reference index of the current block is recorded as DistanceIndexL0, and the distance index of the image corresponding to the L1 reference index of the current block is recorded as DistanceIndexL1; the BlockDistance of the image corresponding to the L0 reference index of the current block is recorded as BlockDistance L0, that is, the distance between the image corresponding to the L0 reference index of the current block and the current frame; the BlockDistance of the image corresponding to the L1 reference index of the current block is denoted as BlockDistance L1, that is, the distance between the image corresponding to the L1 reference index of the current block and the current frame.
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e., the forward motion information of the temporal neighboring block is available), the L0 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef0 (mvRef 0_ x, mvRef0_ y), the distance index of the image where the motion information storage unit is located is denoted as distanceindexcel, and the distance index of the image where the reference unit to which the L0 motion vector points is denoted as distanceindexcef 0. Illustratively, blockDistanceRef0 may be determined by the following equation: blockDistanceRef0= DistanceIndexCol-DistanceIndexRef0.
If the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e., the backward motion information of the temporal neighboring block is available), the L1 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef1 (mvRef 1_ x, mvRef1_ y), the distance index of the image where the motion information storage unit is located is denoted as distanceindexcel, and the distance index of the image where the reference unit to which the L1 motion vector points is denoted as distanceindexcef 1. Illustratively, blockDistanceRef1 may be determined by the following equation: blockDistanceRef1= DistanceIndexCol-DistanceIndexRef1.
Then, let L0 reference index RefIdxL0 of the current block equal to 0, L0 motion vector of the current block be mvE0 (mvE 0_ x, mvE0_ y), L1 reference index RefIdxL1 of the current block equal to 0, L1 motion vector mvE1 of the current block (mvE 1_ x, mvE1_ y). The L0 reference index RefIdxL0 and the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) constitute forward motion information, and the L1 reference index RefIdxL1 and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) constitute backward motion information.
If the reference index L0 stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, and the reference index L1 stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is greater than or equal to 0, the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated by using the formulas (1) and (2), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated by using the formulas (3) and (4).
If the reference index L0 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, and the reference index L1 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is-1, then the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated by using the formulas (1) and (2), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated by using the formulas (5) and (6).
If the reference index of L0 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is-1, and the reference index of L1 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, then the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated by using the formulas (7) and (8), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated by using the formulas (3) and (4).
In embodiment 3, if the current frame is a B frame, the L0 motion information of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with reference index 0 in the reference image queue 1 is located is used as the reference motion information, and the L1 motion information of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with reference index 0 in the reference image queue 1 is located is used as the reference motion information.
Example 4: in order to obtain the temporal motion information of the current block, if the current frame where the current block is located is a P frame, that is, the current picture is a P picture, the manner for determining the temporal motion information is described in embodiment 2, and is not described herein again. If the current frame where the current block is located is a B frame, namely the current image is a B image, determining the time-domain motion information of the current block by adopting the following steps:
step d1, based on the preset position of the current block, determining a time domain adjacent block corresponding to the preset position from the same-position frame, wherein the motion information of the time domain adjacent block comprises forward motion information and backward motion information. And if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to the MVP mode.
Exemplarily, the implementation process of step d1 is similar to that of step c1, and is not described herein again.
And d2, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block.
Illustratively, in comparison with step c2, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is available, the backward motion information in the temporal motion information of the current block is determined according to the forward motion information of the temporal neighboring block, instead of determining the backward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block.
For the process of determining the backward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring block, reference may be made to step c3, which is not repeated herein. For the process of determining the forward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring blocks, reference may be made to step c2, and details are not repeated here.
In another possible embodiment, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is available, the forward motion information in the temporal motion information of the current block is determined according to the backward motion information of the temporal neighboring block, and the backward motion information in the temporal motion information of the current block is determined according to the backward motion information of the temporal neighboring block.
Illustratively, in comparison to step c2, if the forward motion information of the temporal neighboring block is available and the backward motion information of the temporal neighboring block is available, the forward motion information in the temporal motion information of the current block is determined according to the backward motion information of the temporal neighboring block, instead of determining the forward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring block.
For the process of determining the forward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block, reference may be made to step c4, which is not repeated herein. For the process of determining the backward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block, refer to step c2, which is not repeated herein.
And d3, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is unavailable, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block.
Illustratively, the implementation process of step d3 is similar to that of step c3, and is not described herein again.
And d4, if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block.
For example, the implementation process of step d4 is similar to that of step c4, and is not described herein again.
The above process of example 4 is described below with reference to a specific application scenario:
if the L0 reference index and the L1 reference frame index stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located are both-1, then both the L0 reference index and the L1 reference index of the current block are equal to 0. And taking the size and the position of the coding unit where the current block is located as the size and the position of the current block, respectively taking the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to MVP as the L0 motion vector MvE0 and the L1 motion vector MvE1 of the current block, making the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block equal to 0, and ending the derivation process of the time domain motion information of the current block.
And if the L0 reference index and the L1 reference frame index stored in the time domain motion information storage unit of the brightness sample corresponding to the upper-left brightness sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 are not both-1, the L0 reference index and the L1 reference index of the current block are both equal to 0. The distance index of the image corresponding to the L0 reference index of the current block is recorded as DistanceIndexL0, and the distance index of the image corresponding to the L1 reference index of the current block is recorded as DistanceIndexL1; the BlockDistance of the image corresponding to the L0 reference index of the current block is recorded as BlockDistance L0, that is, the distance between the image corresponding to the L0 reference index of the current block and the current frame; the BlockDistance of the picture corresponding to the L1 reference index of the current block is denoted as BlockDistance L1, that is, the distance between the picture corresponding to the L1 reference index of the current block and the current frame.
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e., the forward motion information of the temporal neighboring block is available), the L0 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef0 (mvRef 0_ x, mvRef0_ y), the distance index of the image where the motion information storage unit is located is denoted as distanceindexcel, and the distance index of the image where the reference unit to which the L0 motion vector points is denoted as distanceindexcef 0. Illustratively, blockDistanceRef0 may be determined by the following equation: blockDistanceRef0= DistanceIndexCol-DistanceIndexRef0.
If the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e., the backward motion information of the temporal neighboring block is available), the L1 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef1 (mvRef 1_ x, mvRef1_ y), the distance index of the image where the motion information storage unit is located is denoted as distanceindexcel, and the distance index of the image where the reference unit to which the L1 motion vector points is denoted as distanceindexcef 1. Illustratively, blockDistanceRef1 may be determined by the following equation: blockDistanceRef1= DistanceIndexCol-DistanceIndexRef1.
Then, let L0 reference index RefIdxL0 of the current block equal to 0, L0 motion vector of the current block be mvE0 (mvE 0_ x, mvE0_ y), L1 reference index RefIdxL1 of the current block equal to 0, L1 motion vector mvE1 of the current block (mvE 1_ x, mvE1_ y). The L0 reference index RefIdxL0 and the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) constitute forward motion information, and the L1 reference index RefIdxL1 and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) constitute backward motion information.
If the reference index of L0 stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated by using formulas (1) and (2), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated by using formulas (5) and (6).
If the reference index of L0 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is-1, and the reference index of L1 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, then the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated by using the formulas (7) and (8), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated by using the formulas (3) and (4).
In embodiment 4, if the current frame is a B frame, the L0 motion information of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with reference index 0 in the reference image queue 1 is located is used as the reference motion information, and the L1 motion information of the temporal motion information storage unit in which the luma sample corresponding to the top-left luma sample position of the current block in the image with reference index 0 in the reference image queue 1 is located is used as the reference motion information.
Example 5: in order to obtain the temporal motion information of the current block, if the current frame where the current block is located is a P frame, that is, the current picture is a P picture, the determination method of the temporal motion information is described in embodiment 2, and is not described herein again. If the current frame where the current block is located is a B frame, namely the current image is a B image, determining the time-domain motion information of the current block by adopting the following steps:
step s1, based on the preset position of the current block, determining a time domain adjacent block corresponding to the preset position from the co-located frame, wherein the motion information of the time domain adjacent block comprises forward motion information and backward motion information. And if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to the MVP mode.
And step s2, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block.
For example, the implementation process of step s2 is similar to that of step c2, and is not described herein again.
And step s3, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is unavailable, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block.
For an exemplary process of determining the forward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring block, reference may be made to the process of determining the forward motion information in the temporal motion information in step c2, which is not described again.
And step s4, if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block.
For an exemplary process of determining backward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block, reference may be made to the process of determining backward motion information in the temporal motion information in step c2, which is not described again.
The above process of example 5 is described below with reference to a specific application scenario:
if the L0 reference index and the L1 reference frame index stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located are both-1, then both the L0 reference index and the L1 reference index of the current block are equal to 0. And taking the size and the position of the coding unit where the current block is located as the size and the position of the current block, respectively taking the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to the MVP as an L0 motion vector MvE0 and an L1 motion vector MvE1 of the current block, respectively enabling the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block to be equal to 0, and ending the derivation process of the time domain motion information of the current block.
And if the L0 reference index and the L1 reference frame index stored in the time domain motion information storage unit of the brightness sample corresponding to the upper left brightness sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 are not both-1, the L0 reference index and the L1 reference index of the current block are both equal to 0. The distance index of the image corresponding to the L0 reference index of the current block is recorded as DistanceIndexL0, and the distance index of the image corresponding to the L1 reference index of the current block is recorded as DistanceIndexL1; the BlockDistance of the image corresponding to the L0 reference index of the current block is recorded as BlockDistance L0, that is, the distance between the image corresponding to the L0 reference index of the current block and the current frame; the BlockDistance of the picture corresponding to the L1 reference index of the current block is denoted as BlockDistance L1, that is, the distance between the picture corresponding to the L1 reference index of the current block and the current frame.
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e. the forward motion information of the temporal neighboring block is available), the L0 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef0 (mvRef 0_ x, mv ref0_ y), the distance index of the image where the motion information storage unit is located is denoted as DistanceIndexCol, and the distance index of the image where the reference unit to which the L0 motion vector points is denoted as distanceindexcef 0. Illustratively, blockDistanceRef0 may be determined by the following equation: blockDistanceRef0= DistanceIndexCol-DistanceIndexRef0.
If the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e., the backward motion information of the temporal neighboring block is available), the L1 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef1 (mvRef 1_ x, mvRef1_ y), the distance index of the image where the motion information storage unit is located is denoted as distanceindexcel, and the distance index of the image where the reference unit to which the L1 motion vector points is denoted as distanceindexcef 1. Illustratively, blockDistanceRef1 may be determined by the following equation: blockDistanceRef1= DistanceIndexCol-DistanceIndexRef1.
If the reference index L0 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, and the reference index L1 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is greater than or equal to 0, let the reference index L0 RefIdxL0 of the current block be equal to 0, the L0 motion vector be mvE0 (mvE 0_ x, mvE0_ y), the reference index L1 of L1 be equal to 0, and the motion vector L1 (mvE 1_ x, mvE1_ y). The L0 reference index RefIdxL0 and the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) constitute forward motion information, and the L1 reference index RefIdxL1 and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) constitute backward motion information.
On this basis, the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated using formula (1) and formula (2), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated using formula (3) and formula (4).
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, and the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is-1, the L0 reference index RefIdxL0 of the current block is made equal to 0, the L0 motion vector of the current block is mvE0 (mvE 0_ x, mvE0_ y), the L1 reference index RefIdxL1 of the current block is made equal to-1, and the L1 motion vector of the current block is made mvE1 (0, 0). The L0 reference index RefIdxL0 and the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) constitute forward motion information, and a L1 reference index RefIdxL1 of-1 indicates that the backward motion information is not available.
On this basis, the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated using formula (1) and formula (2).
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is-1, and the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, the L0 reference index RefIdxL0 of the current block is made equal to-1, the L0 motion vector of the current block is mvE0 (0, 0), the L1 reference index RefIdxL1 of the current block is made equal to 0, and the L1 motion vector of the current block is made mvE1 (mvE 1_ x, mvE1_ y). The L1 reference index RefIdxL1 and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) constitute backward motion information, and a L0 reference index RefIdxL0 of-1 indicates that forward motion information is not available.
On this basis, the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated using formula (3) and formula (4).
Example 6: in order to obtain the temporal motion information of the current block, if the current frame where the current block is located is a P frame, that is, the current picture is a P picture, the determination method of the temporal motion information is described in embodiment 2, and is not described herein again. If the current frame where the current block is located is a B frame, namely the current image is a B image, determining the time-domain motion information of the current block by adopting the following steps:
and w1, based on the preset position of the current block, determining a time domain adjacent block corresponding to the preset position from the same-position frame, wherein the motion information of the time domain adjacent block comprises forward motion information and backward motion information. And if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to the MVP mode.
And w2, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block, and determining the backward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block.
Illustratively, as compared to step c2, the backward motion information in the temporal motion information of the current block is determined according to the forward motion information of the temporal neighboring block, instead of determining the backward motion information in the temporal motion information of the current block according to the backward motion information of the temporal neighboring block. For the process of determining the backward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring blocks, refer to step c3, which is not described herein again. For the process of determining the forward motion information in the temporal motion information of the current block according to the forward motion information of the temporal neighboring blocks, reference may be made to step c2, which is not described herein again.
And step w3, if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is unavailable, determining the forward motion information in the time domain motion information of the current block according to the forward motion information of the time domain adjacent block.
And step w4, if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information of the current block according to the backward motion information of the time domain adjacent block.
The above process of example 6 is described below with reference to a specific application scenario:
if the L0 reference index and the L1 reference frame index stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located are both-1, then both the L0 reference index and the L1 reference index of the current block are equal to 0. And taking the size and the position of the coding unit where the current block is located as the size and the position of the current block, respectively taking the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to MVP as the L0 motion vector MvE0 and the L1 motion vector MvE1 of the current block, making the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block equal to 0, and ending the derivation process of the time domain motion information of the current block.
And if the L0 reference index and the L1 reference frame index stored in the time domain motion information storage unit of the brightness sample corresponding to the upper left brightness sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 are not both-1, the L0 reference index and the L1 reference index of the current block are both equal to 0. The distance index of the image corresponding to the L0 reference index of the current block is recorded as DistanceIndexL0, and the distance index of the image corresponding to the L1 reference index of the current block is recorded as DistanceIndexL1; the BlockDistance of the image corresponding to the L0 reference index of the current block is recorded as BlockDistance L0, that is, the distance between the image corresponding to the L0 reference index of the current block and the current frame; the BlockDistance of the picture corresponding to the L1 reference index of the current block is denoted as BlockDistance L1, that is, the distance between the picture corresponding to the L1 reference index of the current block and the current frame.
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e., the forward motion information of the temporal neighboring block is available), the L0 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef0 (mvRef 0_ x, mvRef0_ y), the distance index of the image where the motion information storage unit is located is denoted as distanceindexcel, and the distance index of the image where the reference unit to which the L0 motion vector points is denoted as distanceindexcef 0. Illustratively, blockDistanceRef0 may be determined by the following equation: blockDistanceRef0= DistanceIndexCol-DistanceIndexRef0.
If the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0 (i.e. the backward motion information of the temporal neighboring block is available), the L1 motion vector of the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is denoted as mvRef1 (mv ref1_ x, mv 1_ y), the distance index of the image where the motion information storage unit is located is denoted as DistanceIndexCol, and the distance index of the image where the reference unit to which the L1 motion vector points is denoted as distanceindexcef 1. Illustratively, blockDistanceRef1 may be determined by the following equation: blockDistanceRef1= DistanceIndexCol-DistanceIndexRef1.
If the reference index L0 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, and the reference index L1 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is greater than or equal to 0, let the reference index L0 RefIdxL0 of the current block be equal to 0, the L0 motion vector be mvE0 (mvE 0_ x, mvE0_ y), the reference index L1 of L1 be equal to 0, and the motion vector L1 (mvE 1_ x, mvE1_ y). The L0 reference index RefIdxL0 and the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) constitute forward motion information, and the L1 reference index RefIdxL1 and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) constitute backward motion information.
On this basis, the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated using formula (1) and formula (2), and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated using formula (5) and formula (6).
If the reference index L0 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, and the reference index L1 stored in the temporal motion information storage unit where the luma sample corresponding to the top-left luma sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is-1, then let the reference index L0 RefIdxL0 of the current block be equal to 0, let the motion vector L0 of the current block be mvE0 (mvE 0_ x, mvE0_ y), let the reference index L1 RefIdxL1 of the current block be equal to-1, and let the motion vector L1 of the current block be mvE1 (0, 0). The L0 reference index RefIdxL0 and the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) constitute forward motion information, and a L1 reference index RefIdxL1 of-1 indicates that the backward motion information is not available.
On this basis, the L0 motion vector mvE0 (mvE 0_ x, mvE0_ y) is calculated using formula (1) and formula (2).
If the L0 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is-1, and the L1 reference index stored in the temporal motion information storage unit where the luminance sample corresponding to the top-left luminance sample position of the current block in the image with the reference index value of 0 in the reference image queue 1 is located is greater than or equal to 0, the L0 reference index RefIdxL0 of the current block is made equal to-1, the L0 motion vector of the current block is mvE0 (0, 0), the L1 reference index RefIdxL1 of the current block is made equal to 0, and the L1 motion vector of the current block is made mvE1 (mvE 1_ x, mvE1_ y). The L1 reference index RefIdxL1 and the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) constitute backward motion information, and a L0 reference index RefIdxL0 of-1 indicates that forward motion information is not available.
On this basis, the L1 motion vector mvE1 (mvE 1_ x, mvE1_ y) is calculated using formula (3) and formula (4).
In the above embodiments, the current block may also be referred to as a current prediction unit.
In a possible implementation manner, if the control information allows the current block to enable the temporal motion information derivation mode, the temporal motion information of the current block may be derived by using any of embodiments 1 to 6. Illustratively, the control information may include, but is not limited to: sequence level control information, frame level control information, slice level control information, tile level control information, patch level control information, CTU (Coding Tee Unit) level control information, LCU (Largest Coding Unit) level control information, block level control information, CU (Coding Unit ) level control information, PU (Prediction Unit ) level control information, and the like, without limitation. For example, for the encoding side and the decoding side, if it is known that the control information allows the current block to enable the temporal motion information derivation mode, any of embodiments 1 to 6 is adopted to derive the temporal motion information of the current block.
Example 7: based on the temporal motion information derivation methods in embodiments 1 to 6, the embodiments of the present application provide a coding and decoding method, which can be applied to a coding end or a decoding end. For the encoding end, a motion information candidate list of the current block may be constructed, where the motion information candidate list includes temporal motion information of the current block, and a derivation manner of the temporal motion information of the current block may refer to any embodiment in embodiments 1 to 6, which is not described herein again. After the motion information candidate list is obtained, the coding end sequentially traverses each motion information in the motion information candidate list, determines the rate distortion cost value corresponding to the motion information, and selects the motion information corresponding to the minimum rate distortion cost value. And carrying the position index information of the motion information corresponding to the minimum rate distortion cost value in the motion information candidate list into the coded bit stream of the current block, and transmitting the coded bit stream to a decoding end.
For the decoding end, a motion information candidate list of the current block may also be constructed, where the motion information candidate list includes temporal motion information of the current block, and a derivation manner of the temporal motion information of the current block may refer to any embodiment in embodiments 1 to 6, which is not described herein again. The motion information candidate list constructed by the decoding end is the same as the motion information candidate list constructed by the encoding end. The decoding end can analyze the position index information of the motion information in the motion information candidate list from the coded bit stream of the current block, select the motion information corresponding to the position index information from the motion information candidate list, perform motion compensation on the current block based on the selected motion information to obtain the predicted value of the current block, and does not limit the motion compensation process.
Example 8: the implementation processes of embodiments 1 to 7 may be applied to an AWP (Angular Weighted Prediction) mode, that is, the motion information candidate list may be a motion information candidate list of the AWP mode, the AWP mode sets a reference weight value of a peripheral position (an entire pixel position and a sub-pixel position) of a current block by using an intra-frame angle Prediction idea, obtains a weight value corresponding to each pixel position by using an angle, and implements weighting of two different inter-frame Prediction values by using a finally obtained weight array, where the motion information candidate list is described below with reference to the AWP mode.
Step e1, when determining to start angle weighted prediction on the current block, obtaining the weighted prediction angle of the current block, the weighted prediction position of the current block and the weighted transformation rate of the current block. And determining the initial position of the weight transformation of the current block based on at least one of the weight prediction angle of the current block, the weight prediction position of the current block and the size of the current block. For example, the weight transformation rate of the current block and the starting position of the weight transformation of the current block may be collectively referred to as the weight configuration parameter of the current block.
Illustratively, the weighted prediction angle represents an angular direction pointed by a pixel position inside the current block, and referring to fig. 4A, based on a certain weighted prediction angle, an angular direction pointed by a pixel position inside the current block (such as pixel position 1, pixel position 2, and pixel position 3) is shown, and the angular direction points to a certain peripheral position outside the current block. Referring to FIG. 4B, the angle is predicted based on another weight, which shows the angular direction pointed by the pixel positions (e.g., pixel position 2, pixel position 3, and pixel position 4) inside the current block, and the angular direction points to some peripheral position outside the current block.
The weight prediction angle may be any angle, such as any angle within 180 degrees, or any angle within 360 degrees, and is not limited. For example, the weight prediction angle may be a horizontal angle or a vertical angle, or the absolute value of the slope of the weight prediction angle may be n-th power of 2, where n is an integer, such as a positive integer, 0, a negative integer, or the like. For example, the absolute value of the slope of the weighted prediction angle may be 1 (i.e., 0 of 2), 2 (i.e., 1 of 2), 1/2 (i.e., 1 of 2), and so on. Referring to fig. 4C, 8 weighted prediction angles are shown, the absolute value of the slope of which is 2 to the nth power.
The weight prediction position is used for configuring a reference weight value of the peripheral position outside the current block. For example, a range of peripheral positions outside the current block (i.e., the number of peripheral positions outside the current block) is determined according to the weight prediction angle of the current block, the size of the current block, and the like, the range of the peripheral positions is divided into N equal parts, the values of N are arbitrarily arranged, such as 4, 6, 8, and the like, and 8 is taken as an example to illustrate, and the weight prediction position is used to indicate which peripheral position outside the current block is a start position of weight transformation of the current block, so that a reference weight value of the peripheral position outside the current block is arranged according to the start position of the weight transformation. Referring to fig. 4D, after dividing all the peripheral positions 8 equally, 7 weighted predicted positions can be obtained. On this basis, when the weight prediction position is 0, the peripheral position a0 can be represented as the starting position of the weight transformation of the peripheral position outside the current block, and so on.
The weight transformation rate indicates a transformation rate of the reference weight value of the peripheral position outside the current block for indicating a change speed of the reference weight value, and may be any number other than 0, e.g., the weight transformation rate may be-4, -2, -1, 2, 4, 0.5, 0.75, 1.5, etc. When the absolute value of the weight conversion ratio is 1, that is, the weight conversion ratio is-1 or 1, it is indicated that the change speed of the reference weight value is 1, and based on this, the reference weight value needs to go through 0,1,2,3,4,5,6,7,8, etc. from 0 to 8, and the reference weight value needs to go through 8,7,6,5,4,3,2,1,0, etc. from 8 to 0.
And e2, configuring a reference weight value for the peripheral position outside the current block according to the weight configuration parameter of the current block.
For example, the number of peripheral positions outside the current block may be determined based on the size of the current block and/or the weighted prediction angle of the current block, for example, the number M of peripheral positions outside the current block is determined based on the size of the current block and/or the weighted prediction angle of the current block, and the reference weight values are configured for the M peripheral positions according to the weight configuration parameters of the current block.
For example, the peripheral locations outside the current block may include integer-pixel locations, or sub-pixel locations, or both integer-pixel and sub-pixel locations. The peripheral locations outside the current block may include, but are not limited to: the peripheral position of the upper line outside the current block, or the peripheral position of the left column outside the current block, or the peripheral position of the lower line outside the current block, or the peripheral position of the right column outside the current block. Of course, the above is only an example of the peripheral position, and the present invention is not limited thereto.
In step e2, an effective number (the number of peripheral positions outside the current block is an effective number) of reference weight values needs to be obtained first, where the effective number may be determined based on the size of the current block and/or the weight prediction angle of the current block, and for example, the effective number may be determined as follows: validLenth = (N + (M > > X)) < <1, N and M are height and width of the current block, respectively, and X is log2 log of absolute value of slope of weight prediction angle of the current block, such as 0 or 1.
The reference weight values may be monotonically increasing, or monotonically decreasing, for a significant number of reference weight values. For example, an effective number of reference weight values may be [88.. 88765432100.. 00], i.e., monotonically decreasing. As another example, the effective number of reference weight values may be [00.. 00123456788.. 88], i.e., monotonically increasing. Of course, the above is merely an example, and no limitation is made thereto.
Referring to fig. 4E, there are several exemplary cases of an effective number of reference weight values, which are not limited thereto.
In one possible implementation, an effective number of reference weight values may be determined according to the weight transformation ratio and the start position of the weight transformation. For example, the reference weight value may be determined as follows: y = Clip3 (minimum, maximum, a × (x-s)), x represents the index of the peripheral position, i.e. x may range from 1-significant value, e.g. x is 1, representing the 1 st peripheral position, y represents the reference weight value of the 1 st peripheral position, x is 2, representing the 2 nd peripheral position, y represents the reference weight value of the 2 nd peripheral position. a is used to represent the weight transformation ratio and s is used to represent the start position of the weight transformation.
Clip3 is used to limit the reference weight value between a minimum value and a maximum value, both of which may be configured empirically, and for convenience of description, in the following process, the minimum value is 0, and the maximum value is 8 for example.
s denotes the starting position of the weight transformation, and s can be determined from the weight prediction position, e.g. s = f (weight prediction position), i.e. s is a function related to the weight prediction position. For example, after the range of the peripheral positions outside the current block is determined, the effective number of the peripheral positions may be determined, and all the peripheral positions are divided into N equal parts, where the value of N may be arbitrarily configured, such as 4, 6, 8, etc., and the weighted prediction position is used to indicate which peripheral position outside the current block is used as the target peripheral region of the current block, and the peripheral position corresponding to the weighted prediction position is the starting position of the weighted transformation. Alternatively, s may be determined by a weight prediction angle and a weight prediction position, for example, s = f (weight prediction angle, weight prediction position), i.e., s is a function related to the weight prediction angle and the weight prediction position. For example, the range of the peripheral position outside the current block may be determined according to the weighted prediction angle, after the range of the peripheral position outside the current block is determined, the effective number of the peripheral positions may be determined, and all the peripheral positions may be divided into N equal parts, the weighted prediction position is used to indicate which peripheral position outside the current block is used as the target peripheral region of the current block, and the peripheral position corresponding to the weighted prediction position may be the start position of the weighted transformation.
In summary, an effective number of reference weight values for the current block may be obtained, which are monotonically increasing or monotonically decreasing. Then, according to the effective number of reference weight values, reference weight values of peripheral positions outside the current block are configured. For example, the number of the peripheral locations outside the current block is a significant number, and the number of the reference weight values is a significant number, so that the significant number of the reference weight values may be configured as the reference weight values of the peripheral locations outside the current block.
E3, aiming at each pixel position of the current block, determining a peripheral matching position pointed by the pixel position from peripheral positions outside the current block according to the weight prediction angle of the current block; and determining a target weight value of the pixel position according to the reference weight value associated with the peripheral matching position, and determining an associated weight value of the pixel position according to the target weight value of the pixel position.
For example, since the weighted prediction angle represents an angular direction pointed to by a pixel position inside the current block, for each pixel position of the current block, the angular direction pointed to by the pixel position is determined based on the weighted prediction angle, and then a peripheral matching position pointed to by the pixel position is determined from peripheral positions outside the current block according to the angular direction.
For each pixel position of the current block, after determining the peripheral matching position pointed by the pixel position, determining a target weight value of the pixel position based on the reference weight value associated with the peripheral matching position, for example, determining the reference weight value associated with the peripheral matching position as the target weight value of the pixel position. Then, the associated weight value of the pixel position is determined according to the target weight value of the pixel position, for example, the sum of the target weight value and the associated weight value of each pixel position may be a fixed preset value, and therefore, the associated weight value may be the difference between the preset value and the target weight value. Assuming that the preset value is 8 and the target weight value of the pixel position is 0, the associated weight value of the pixel position is 8; if the target weight value of a pixel position is 1, the associated weight value of the pixel position is 7, and so on, as long as the sum of the target weight value and the associated weight value is 8.
Step e4, obtaining a motion information candidate list, wherein the motion information candidate list comprises at least one candidate motion information; first target motion information and second target motion information of the current block are acquired based on the motion information candidate list.
For the encoding end, based on the rate distortion principle, one candidate motion information may be selected from the motion information candidate list as the first target motion information of the current block, and another candidate motion information may be selected from the motion information candidate list as the second target motion information of the current block, where the first target motion information is different from the second target motion information, and this is not limited.
When the encoding end sends the encoding bit stream to the decoding end, the encoding bit stream carries indication information a and indication information b, the indication information a is used for indicating an index value 1 of the first target motion information of the current block, and the index value 1 indicates that the first target motion information is the several candidate motion information in the motion information candidate list. The indication information b is used to indicate an index value 2 of the second target motion information of the current block, and the index value 2 indicates that the second target motion information is the several candidate motion information in the motion information candidate list.
For the decoding end, after receiving the coded bit stream, the indication information a and the indication information b are analyzed from the coded bit stream. Based on the indication information a, the decoding end selects candidate motion information corresponding to the index value 1 from the motion information candidate list, and the candidate motion information is used as the first target motion information of the current block. Based on the indication information b, the decoding end selects candidate motion information corresponding to the index value 2 from the motion information candidate list, and the candidate motion information is used as second target motion information of the current block.
E5, aiming at each pixel position of the current block, determining a first predicted value of the pixel position according to the first target motion information of the current block, and determining a second predicted value of the pixel position according to the second target motion information of the current block; and determining the weighted predicted value of the pixel position according to the first predicted value, the target weight value, the second predicted value and the associated weight value.
For example, assuming that the target weight value is a weight value corresponding to the first target motion information, and the associated weight value is a weight value corresponding to the second target motion information, the weighted prediction value of the pixel position may be: (the first predicted value of the pixel position + the target weight value of the pixel position + the second predicted value of the pixel position + the associated weight value of the pixel position)/a fixed preset value. Or, assuming that the target weight value is a weight value corresponding to the second target motion information, and the associated weight value is a weight value corresponding to the first target motion information, the weighted prediction value of the pixel position may be: (the second predicted value of the pixel position + the target weight value of the pixel position + the first predicted value of the pixel position + the associated weight value of the pixel position)/a fixed preset value.
And e6, determining the weighted prediction value of the current block according to the weighted prediction values of all the pixel positions of the current block.
For example, the weighted prediction values of all pixel positions of the current block are composed into the weighted prediction value of the current block.
To sum up, for the encoding end and the decoding end, in step e4, the motion information candidate list needs to be obtained, that is, the motion information candidate list of the current block needs to be constructed. For example, a motion information candidate list of the current block is constructed based on available motion information to be added to the motion information candidate list, where the available motion information includes temporal motion information of the current block, and a derivation manner of the temporal motion information of the current block may refer to any embodiment in embodiments 1 to 6, which is not described herein again. In addition to temporal motion information, the available motion information includes at least one of the following motion information: spatial domain motion information; HMVP (History-based Motion Vector Prediction) Motion information; and presetting motion information.
For example, if the available motion information includes spatial motion information, before constructing the motion information candidate list of the current block based on the available motion information to be added to the motion information candidate list, for a spatial neighboring block of the current block, if the spatial neighboring block exists and the spatial neighboring block employs an inter prediction mode, the motion information of the spatial neighboring block may be determined as the available motion information; and/or, for a first spatial neighboring block of the current block, if the first spatial neighboring block exists and the first spatial neighboring block adopts inter prediction mode, and the motion information of the first spatial neighboring block is different from the motion information of a second spatial neighboring block (which may be at least one second spatial neighboring block) of the current block, the motion information of the second spatial neighboring block having been determined to be available motion information, the motion information of the first spatial neighboring block may be determined to be available motion information.
Exemplarily, if the available motion information includes the preset motion information, the preset motion information is determined as the available motion information before constructing the motion information candidate list of the current block based on the available motion information to be added to the motion information candidate list; the preset motion information may include, but is not limited to, at least one of the following motion information: zero motion information; default motion information derived based on motion information already present in the motion information candidate list; motion information already existing in the motion information candidate list.
In one possible embodiment, the motion information candidate list of the current block is constructed based on available motion information to be added to the motion information candidate list, including but not limited to: for available motion information to be added to the motion information candidate list,
if the available motion information is unidirectional motion information, adding the unidirectional motion information into a motion information candidate list; if the available motion information is bidirectional motion information, the bidirectional motion information can be cut into first unidirectional motion information and second unidirectional motion information, and the first unidirectional motion information is added into a motion information candidate list; alternatively, the first and second electrodes may be,
if the available motion information is unidirectional motion information and the unidirectional motion information is not repeated with motion information existing in the motion information candidate list, adding the unidirectional motion information into the motion information candidate list; if the available motion information is bidirectional motion information, cutting the bidirectional motion information into first unidirectional motion information and second unidirectional motion information, and adding certain unidirectional motion information into a motion information candidate list; for example, if the first unidirectional motion information is not overlapped with the motion information existing in the motion information candidate list, adding the first unidirectional motion information into the motion information candidate list; alternatively, the first and second liquid crystal display panels may be,
if the available motion information is unidirectional motion information and the unidirectional motion information is not repeated with motion information existing in the motion information candidate list, adding the unidirectional motion information into the motion information candidate list; if the available motion information is bidirectional motion information, cutting the bidirectional motion information into first unidirectional motion information and second unidirectional motion information, and adding certain unidirectional motion information into a motion information candidate list; for example, if the first unidirectional motion information is not overlapped with the motion information existing in the motion information candidate list, adding the first unidirectional motion information into the motion information candidate list; and if the first unidirectional motion information is repeated with the motion information already existing in the motion information candidate list and the second unidirectional motion information is not repeated with the motion information already existing in the motion information candidate list, adding the second unidirectional motion information into the motion information candidate list.
In the above embodiment, the first unidirectional motion information may be unidirectional motion information pointing to a reference frame in the first reference frame list; the second unidirectional motion information may be unidirectional motion information that points to a reference frame in the second reference frame list.
For example, if the total number of candidate motion information existing in the motion information candidate list (i.e., the total number currently existing in the motion information candidate list) is an even number, the first reference frame list is a forward reference frame list of the current block, and the second reference frame list is a backward reference frame list of the current block; if the total number of the candidate motion information existing in the motion information candidate list is an odd number, the first reference frame list is a backward reference frame list of the current block, and the second reference frame list is a forward reference frame list of the current block. Or, if the total number of the candidate motion information existing in the motion information candidate list is an odd number, the first reference frame list is a forward reference frame list of the current block, and the second reference frame list is a backward reference frame list of the current block; if the total number of the candidate motion information existing in the motion information candidate list is an even number, the first reference frame list is a backward reference frame list of the current block, and the second reference frame list is a forward reference frame list of the current block.
Illustratively, the first reference frame list is a forward reference frame list of the current block, and the second reference frame list is a backward reference frame list of the current block. Or, the first reference frame list is a backward reference frame list of the current block, and the second reference frame list is a forward reference frame list of the current block.
The following describes a process for constructing a motion information candidate list in the AWP mode, with reference to several application scenarios. In each application scenario, the motion information candidate list is referred to as awwpcandarray, and assuming that the length of the awwpcandarray is X, that is, X pieces of available motion information need to be added, the X may be configured empirically, for example, the length of the awwpcandarray is 5.
Application scenario 1: the motion information candidate list for the AWP mode can be constructed by:
step F1, as shown in fig. 4F, G, C, a, B, and D are spatial neighboring blocks of the current block, and the "availability" of the motion information of F, G, C, a, B, and D can be determined. Illustratively, if F exists and the inter prediction mode is adopted, the motion information of F is available motion information; otherwise, the motion information of F is unavailable motion information. If G exists and the inter-frame prediction mode is adopted, the motion information of G is available motion information; otherwise, the motion information of G is unavailable motion information. If C exists and the inter-frame prediction mode is adopted, the motion information of C is available motion information; otherwise, the motion information of C is unavailable motion information. If A exists and the inter-frame prediction mode is adopted, the motion information of A is available motion information; otherwise, the motion information of a is unavailable motion information. If B exists and the inter-frame prediction mode is adopted, the motion information of B is available motion information; otherwise, the motion information of B is unavailable motion information. If D exists and the inter-frame prediction mode is adopted, the motion information of D is available motion information; otherwise, the motion information of D is unavailable motion information.
And step F2, adding the unidirectional available motion information to the AwpCandArray and repeating the information according to the sequence of F, G, C, A, B and D (the sequence is variable) until the length of the AwpCandArray is Y or the traversal is finished. The value of Y may be smaller than that of X, and may be configured according to experience, where Y is 4, which indicates that at least X-Y positions are reserved for the time-domain motion information.
And F3, if the length of the AfwCandArray is smaller than Y, splitting the bidirectional available motion information into unidirectional motion information pointing to a forward reference frame List (namely a reference frame List 0) and unidirectional motion information pointing to a backward reference frame List (namely a reference frame List List 1) according to the sequence of F, G, C, A, B and D (the sequence is variable), firstly carrying out a check operation on the unidirectional motion information, and if the unidirectional motion information is not repeated, putting the AfwCandArray into the check operation and checking the check operation until the length of the AfwCandArray is Y or the end of the traversal.
And f4, splitting the time domain motion information of the current block into unidirectional motion information pointing to a forward reference frame list and unidirectional motion information pointing to a backward reference frame list, performing unidirectional motion information duplication checking operation, and if the unidirectional motion information is not repeated, putting the unidirectional motion information into the AwpCandArray until the length of the AwpCandArray is X or the traversal is finished.
For example, see embodiments 1-6 regarding the way of deriving temporal motion information of a current block.
And f5, if the length of the AwpCandArray is smaller than X, repeatedly filling the last unidirectional motion information in the AwpCandArray until the length of the AwpCandArray is X, and obtaining the AwpCandArray of the current block.
Application scenario 2: the motion information candidate list for the AWP mode can be constructed by:
step G1, F, G, C, A, B and D are spatial adjacent blocks of the current block, if F exists and an inter-frame prediction mode is adopted, the motion information of F is available motion information; otherwise, the motion information of F is unavailable motion information. If G exists and the inter-frame prediction mode is adopted, and the motion information of G is different from that of F, the motion information of G is available motion information; otherwise, the motion information of G is unavailable motion information. If the C exists and adopts an inter-frame prediction mode, and the motion information of the C is different from the motion information of the G, the motion information of the C is available motion information; otherwise, the motion information of C is unavailable motion information. If A exists and the inter-frame prediction mode is adopted, the motion information of A is different from the motion information of F, and the motion information of A is available motion information; otherwise, the motion information of a is unavailable motion information. If B exists and the inter-frame prediction mode is adopted, the motion information of B is available motion information; otherwise, the motion information of B is unavailable motion information. If D exists and an inter-frame prediction mode is adopted, the motion information of D is different from the motion information of A, and the motion information of D is different from the motion information of G, the motion information of D is available motion information; otherwise, the motion information of D is unavailable motion information.
And step G2, putting the available motion information (unidirectional motion information or bidirectional motion information) into the AwpCandArray in the sequence of F, G, C, A, B and D (the sequence is variable) until the length of the AwpCandArray is Y or the traversal is finished. The value of Y may be smaller than the value of X, and may be configured according to experience, for example, the value of Y is 4, which indicates that at least X-Y positions are reserved for the time domain motion information.
And step g3, putting the time domain motion information of the current block into the AwpCandArray until the length of the AwpCandArray is X or the traversal is finished.
For example, see embodiments 1-6 regarding the way in which temporal motion information of a current block is derived.
And g4, if the length of the AwpCandArray is smaller than X, adding bidirectional zero motion information to the AwpCandArray until the length of the AwpCandArray is X to obtain the AwpCandArray of the current block. In the bidirectional zero motion information, the motion vector value may be zero, the L0 reference picture index may be equal to 0, and the L1 reference picture index may be equal to 0.
Step g5, assigning one unidirectional motion information of the AwpCandIdx0+1 motion information in the AwpCandArray to mvAwp0L0, mvAwp0L1, refIdxAwp0L0 and RefIdxAwp0L1, and if the assignment is carried out in the following mode:
case 1, if awpcandxdx 0 is even, then assign uni-directional motion information pointing to the forward reference frame list to mvAwp0L0 and RefIdxAwp0L0, mvAwp0L1 equals zero, and RefIdxAwp0L1 equals-1. If the one-way motion information pointing to the forward reference frame list does not exist, the one-way motion information pointing to the backward reference frame list is assigned to mvAwp0L1 and RefIdxawp0L1, mvAwp0L0 is equal to zero, and RefIdxawp0L0 is equal to-1.
Case 2, if awpcandiddx 0 is odd, assign the one-way motion information pointing to the backward reference frame list to mvAwp0L1 and RefIdxAwp0L1, mvAwp0L0 equals zero, and RefIdxAwp0L0 equals-1. If unidirectional motion information pointing to the backward reference frame list does not exist, unidirectional motion information pointing to the forward reference frame list is assigned to mvAwp0L0 and RefIdxawp0L0, mvAwp0L1 is equal to zero, refIdxawp0L1 is equal to-1.
Step g6, assigning one unidirectional motion information of the AwpCandIdx1+1 motion information in the AwpCandArray to mvAwp1L0, mvAwp1L1, refIdxAWp1L0 and RefIdxAWp1L1, and if the assignment is carried out in the following mode:
case 1, if awpcandxdx 1 is even, then assign uni-directional motion information pointing to the forward reference frame list to mvAwp1L0 and RefIdxAwp1L0, mvAwp1L1 equals zero, and RefIdxAwp1L1 equals-1. If the one-way motion information pointing to the forward reference frame list does not exist, the one-way motion information pointing to the backward reference frame list is assigned to mvAwp1L1 and RefIdxawp1L1, mvAwp1L0 is equal to zero, and RefIdxawp1L0 is equal to-1.
Case 2, if awpcandiddx 1 is odd, assign the one-way motion information pointing to the backward reference frame list to mvAwp1L1 and RefIdxAwp1L1, mvAwp1L0 equals zero, and RefIdxAwp1L0 equals-1. If unidirectional motion information pointing to the backward reference frame list does not exist, unidirectional motion information pointing to the forward reference frame list is assigned to mvAwp1L0 and RefIdxawp1L0, mvAwp1L1 is equal to zero, and RefIdxawp1L1 is equal to-1.
Application scenario 3: the motion information candidate list for the AWP mode can be constructed by:
the implementation process of steps h1 to h4 is the same as that of steps f1 to f4, and will not be repeated herein.
And step h5, if the length of the AfwpCandArray is smaller than X, sequentially generating four unidirectional motion information, wherein a motion vector corresponding to the first unidirectional motion information in the AfwpCandArray is (X, y), reference frame information of the generated four unidirectional motion information is the same as reference frame information of the first unidirectional motion information in the AfwpCandArray, motion vectors corresponding to the four unidirectional motion information are respectively (X0, y 0), (X1, y 1), (X2, y 2), (X3, y 3), and generation modes of the motion vectors are shown in the following modes. And adding the newly generated motion information into the AwpCandArray in sequence until the length of the AwpCandArray is X.
x0＝abs(x)<88:(abs(x)<＝64？((abs(x)*5+2)>>2):((abs(x)*9+4)>>3))
x0＝x<0？-x0:x0
y0＝y
x1＝x
y1＝abs(y)<88:(abs(y)<＝64？((abs(y)*5+2)>>2):((abs(y)*9+4)>>3))
y1＝y<0？-y1:y1
x2＝abs(x)<88:(abs(x)<＝64？((abs(x)*3+2)>>2):((abs(x)*7+4)>>3))
x2＝x<＝0？-x2:x2
y2＝y
x3＝x
y3＝abs(y)<88:(abs(y)<＝64？((abs(y)*3+2)>>2):((abs(y)*7+4)>>3))
y3＝y<＝0？-y3:y3
And h6, if the length of the AwpCandArray is smaller than X, repeatedly filling the last unidirectional motion information in the AwpCandArray until the length of the AwpCandArray is X, and obtaining the AwpCandArray of the current block.
Application scenario 4: the motion information candidate list for the AWP mode can be constructed by:
the implementation process of step k1 to step k3 is the same as that of step g1 to step g3, and will not be repeated herein.
And k4, if the length of the AwpCandArray is smaller than X, sequentially generating four pieces of unidirectional motion information, and sequentially adding the newly generated motion information into the AwpCandArray until the length of the AwpCandArray is X. If the first valid motion information in the awwpcandarray is the one-way motion information, and the motion vector is marked as (x, y), the reference frame information of the generated four one-way motion information is the same as the reference frame information of the first valid motion information, and the motion vectors of the generated four one-way motion information are respectively (x 0, y 0), (x 1, y 1), (x 2, y 2), and (x 3, y 3), which is referred to step h5 specifically, and is not described herein again.
If the first valid motion information in the awwpcandarray is bidirectional motion information, the motion vectors are marked as (x, y) and (x ', y'), and the generated reference frame information of the four pieces of bidirectional motion information is the same as the reference frame information of the first valid motion information. The motion vectors of the generated four pieces of bidirectional motion information are (x 0, y 0) and (x 0', y 0'), (x 1, y 1) and (x 1', y 1'), (x 2, y 2) and (x 2', y 2'), (x 3, y 3) and (x 3', y 3'), respectively. See step h5 for determination of (x 0, y 0), (x 1, y 1), (x 2, y 2), (x 3, y 3). (x 0', y 0'), (x 1', y 1'), (x 2', y 2'), and (x 3', y 3') are determined as follows:
x0’＝abs(x’)<88:(abs(x’)<＝64？((abs(x’)*5+2)>>2):((abs(x’)*9+4)>>3))
x0＝x’<0？-x0’:x0’
y0’＝y’
x1’＝x’
y1’＝abs(y’)<88:(abs(y’)<＝64？((abs(y’)*5+2)>>2):((abs(y’)*9+4)>>3))
y1’＝y’<0？-y1’:y1’
x2’＝abs(x’)<88:(abs(x’)<＝64？((abs(x’)*3+2)>>2):((abs(x’)*7+4)>>3))
x2’＝x’<＝0？-x2’:x2’
y2’＝y’
x3’＝x’
y3’＝abs(y’)<88:(abs(y’)<＝64？((abs(y’)*3+2)>>2):((abs(y’)*7+4)>>3))
y3’＝y’<＝0？-y3’:y3’
and step k5, if the length of the AwpCandArray is smaller than X, adding bidirectional zero motion information to the AwpCandArray until the length of the AwpCandArray is X, and obtaining the AwpCandArray of the current block. In the bidirectional zero motion information, the motion vector value may be zero, the L0 reference picture index may be equal to 0, and the L1 reference picture index may be equal to 0.
And step k6, assigning one unidirectional motion information of the AwpCandididx 0+1 motion information in the AwpCandArray to mvAwp0L0, mvAwp0L1, refIdxAWp0L0 and RefIdxAWp0L1, wherein the concrete mode is shown in step g5.
And k7, assigning one unidirectional motion information of the AwpCandIdx1+1 motion information in the AwpCandArray to mvAwp1L0, mvAwp1L1, refIdxAWp1L0 and RefIdxAWp1L1, wherein the specific mode is shown in the step g6.
Application scenario 5: for any of the application scenarios 1-4, when the available motion information is placed into the awwpcandarray in the order of F, G, C, a, B, and D, the length of the awwpcandarray may be X or the traversal may be ended. For example, if the length of the ahwpcandaray is X, the time-domain motion information of the current block may no longer be placed in the ahwpcandaray, that is, the ahwpcandaray may not include the time-domain motion information of the current block. And if the length of the AwpCandArray is smaller than X at the end of traversal, putting the time-domain motion information of the current block into the AwpCandArray.
Application scenario 6: assuming that the minimum block size supported by the AWP mode is 8 and the maximum block size is 64, 8 weighted prediction angles are supported in total, as shown in fig. 4C, and each weighted prediction angle supports 7 weighted prediction positions, as shown in fig. 4D, and thus, for each block, there are 56 modes in total. Referring to fig. 4E, a diagram of reference weights for predicting positions for AWP mode supported weights is shown. Referring to fig. 4G, the weighted prediction angle is divided into 4 partitions, and the formula derived from the pixel-by-pixel weights is slightly different according to the different regions where the weighted prediction angles are located, where the block size of the current block is MxN, M is width, N is height, X is log2 (absolute value of slope of the weighted prediction angle), and Y is the weighted prediction position.
Taking the weighted prediction angle 0 and the weighted prediction angle 1 in the angle partition 0 as an example, the derivation process is as follows:
the effective length ValidLenth is calculated using the following formula: validLenth = (N + (M > > X)) < <1.
Setting a reference weight value ReferenceWeights [ x ], wherein the value range of x is 0-ValidLength-1; firstPos = (ValidLength > > 1) -6+ Y ((ValidLength-1) > > 3); referenceWeights [ x ] = Clip3 (0,8,x-FirstPos); illustratively, firstPos indicates the start position of the weight transform in the above embodiment. Pixel-by-pixel derivation of weights SampleWeight [ x ] [ y ]: sampleWeight [ X ] [ y ] = referenceWeights [ (y < < 1) + ((X < < 1) > > X) ]
The chroma weight is derived as: directly taking the position of the upper left corner corresponding to 2 × 2 brightness weight, wherein the sampling weight chroma [ x ] [ y ] = sampling weight [ x > >1] [ y > >1], the value range of x is 0 to (M/2-1), and the value range of y is 0 to (N/2-1).
Example 9: the implementation processes of embodiment 1 to embodiment 7 may be applied to a UMVE (advanced Motion information Expression) mode, that is, the Motion information candidate list may be a Motion information candidate list of a UMVE mode, where the UMVE mode is to perform offset on the basis of generated Motion information so as to obtain better Motion information, and the Motion information candidate list is described below in combination with the Motion information candidate list of the UMVE mode.
For example, for the encoding end and the decoding end, a motion information candidate list needs to be obtained, that is, a motion information candidate list of the current block needs to be constructed. For example, a motion information candidate list of the current block is constructed based on available motion information to be added to the motion information candidate list. For example, the available motion information includes temporal motion information of the current block, and a derivation manner of the temporal motion information of the current block may refer to any embodiment of embodiments 1 to 6, which is not described herein again. In addition to temporal motion information, the available motion information includes at least one of the following motion information: spatial domain motion information; and presetting motion information.
For example, if the available motion information includes spatial motion information, before constructing the motion information candidate list of the current block based on the available motion information to be added to the motion information candidate list, for a spatial neighboring block of the current block, if the spatial neighboring block exists and the spatial neighboring block adopts an inter prediction mode, the motion information of the spatial neighboring block may be determined as the available motion information; and/or, for a first spatial neighboring block of the current block, if the first spatial neighboring block exists and the first spatial neighboring block adopts inter prediction mode, and the motion information of the first spatial neighboring block is different from the motion information of a second spatial neighboring block (which may be at least one second spatial neighboring block) of the current block, the motion information of the second spatial neighboring block having been determined to be available motion information, the motion information of the first spatial neighboring block may be determined to be available motion information.
Exemplarily, if the available motion information includes the preset motion information, the preset motion information is determined as the available motion information before constructing the motion information candidate list of the current block based on the available motion information to be added to the motion information candidate list; the preset motion information may include, but is not limited to, at least one of the following motion information: zero motion information; default motion information derived based on motion information already present in the motion information candidate list; motion information already existing in the motion information candidate list.
In one possible embodiment, the building of the motion information candidate list of the current block based on the available motion information to be added to the motion information candidate list may include, but is not limited to: for available motion information to be added to a motion information candidate list, if the available motion information is unidirectional motion information, adding the unidirectional motion information to the motion information candidate list; if the available motion information is bidirectional motion information, adding the bidirectional motion information into a motion information candidate list.
In summary, for the construction process of the motion information candidate list in the UMVE mode, the motion information (which needs to be available motion information) of the spatial neighboring block of the current block may be put in first; then, the time domain motion information of the current block is put in; then, zero motion information is put in, and the following describes the process of constructing a motion information candidate list in the UMVE mode in combination with a specific application scenario. Let the motion information candidate list in the UMVE mode be referred to as a UmveList, and assuming that the length of the UmveList is X, i.e. X available motion information needs to be added, X may be configured empirically, for example, the length of the UmveList may be 2.
Step t1, see fig. 4F, where F, G, C, a, and D are spatial neighboring blocks of the current block, and the "availability" of F, G, C, a, and D is determined in sequence according to the order of F- > G- > C- > a- > D. If the F exists and the inter-frame prediction mode is adopted, the motion information of the F is available motion information; otherwise, the motion information of F is unavailable motion information. If G exists and the inter-frame prediction mode is adopted, the motion information of G is different from that of F, and the motion information of G is available motion information; otherwise, the motion information of G is unavailable motion information. If the C exists and adopts an inter-frame prediction mode, and the motion information of the C is different from the motion information of the G, the motion information of the C is available motion information; otherwise, the motion information of C is unavailable motion information. If A exists and the inter-frame prediction mode is adopted, the motion information of A is different from the motion information of F, and the motion information of A is available motion information; otherwise, the motion information of A is unavailable motion information. If D exists and an inter-frame prediction mode is adopted, the motion information of D is different from the motion information of A, and the motion information of D is different from the motion information of G, the motion information of D is available motion information; otherwise, the motion information of D is unavailable motion information.
And step t2, sequentially adding the available airspace motion information into the UmveList according to the sequence of F- > G- > C- > A- > D (the sequence can be changed according to the actual situation) until the length of the UmveList is X or the traversal is finished.
And step t3, if the length of the UmveList is less than X, adding the time-domain motion information of the current block (see embodiment 1-embodiment 6 in a derivation way) into the UmveList until the length of the UmveList is X or the traversal is finished.
And step t4, if the length of the UmveList is less than X, adding zero motion information into the UmveList until the length is X.
Illustratively, based on the motion information candidate list of the UMVE mode, the prediction process of the UMVE mode includes:
referring to fig. 5A, each initial motion information constructed by the motion information candidate list is used as a starting point, and searching is performed from four directions, i.e., up, down, left, and right, with offsets of 1/4, 1/2, 1,2, and 4 pixels, respectively, and the starting point has two (2 base), four offset directions (4 directions), and 5 offsets, thereby generating 40 candidate motion vectors in total.
The encoding end performs motion compensation on the generated candidate motion vector in an RDO manner to obtain an optimal prediction block, and encodes motion information corresponding to the optimal prediction block, including umve _ mv _ idx (indicating initial motion information), umve _ step _ idx (indicating an offset value), and umve _ dir _ idx (indicating an offset direction). The decoding end obtains the motion vector of the current block by constructing a motion information candidate list, resolving umve _ mv _ idx (indicating initial motion information), umve _ step _ idx (indicating offset value) and umve _ dir _ idx (indicating offset direction), and performing motion compensation to obtain the predicted value of the current block.
Example 10: embodiment 1-implementation of embodiment 7 may be applied in AFFINE (AFFINE motion compensation) mode, i.e. the motion information candidate list may be a motion information candidate list of AFFINE mode. Illustratively, the motion information of the AFFINE mode is obtained by calculating the motion information of the top left corner and the top right corner of the current block, the motion information of the current block is obtained by calculating a multi-parameter model, and the AFFINE mode has a good prediction effect on a scene with rotation and zooming.
For example, for an encoding end and a decoding end, it is necessary to obtain a motion information candidate list of the AFFINE mode, that is, it is necessary to construct a motion information candidate list of a current block. For example, a multi-parameter model is obtained based on available motion information, which may include at least four parameters, the available motion information including temporal motion information of the current block and motion information of spatial neighboring blocks of the current block; then, a motion information candidate list of the current block may be constructed according to the multi-parameter model. The deriving manner of the temporal motion information of the current block may refer to any embodiment of embodiments 1 to 6, which is not described herein again.
In one possible embodiment, the multi-parameter model may include a four-parameter model and/or a six-parameter model; if the multi-parameter model includes a four-parameter model, the four-parameter model may be obtained based on temporal motion information of the current block and motion information of one spatial neighboring block. Of course, a four-parameter model may also be obtained based on the motion information of two spatial neighboring blocks. If the multi-parameter model includes a six-parameter model, the six-parameter model may be obtained based on the temporal motion information of the current block and the motion information of the two spatial neighboring blocks. Of course, a four-parameter model may also be obtained based on the motion information of the three spatial neighboring blocks.
For example, the AFFINE mode employs a multi-parameter model, such as a four-parameter model and a six-parameter model, and the motion information of each sub-block is obtained through the motion information of two or three control points, as shown in fig. 5B. The AFFINE mode can be divided into two parts, AFFINE skip/direct and AFFINE inter. The AFFINE skip/direct derives a multi-parameter model using some information around the current block, and builds a motion information candidate list based on the multi-parameter model, and the motion information candidate list can support 5 multi-parameter models. The construction process of the motion information candidate list of the AFFINE skip/direct comprises the following steps: copying the multi-parameter model from an adjacent AFFINE coding unit; generating a multi-parameter model by utilizing the motion information of angular points around the current block; if the number of multi-parameter models of the motion information candidate list does not reach five, zero is padded. The AFFINE inter is added with MVD on the basis of AFFINE skip/direct, each control point needs to transmit the MVD, and the construction process of the motion information candidate list is similar.
The following describes a process of constructing a motion information candidate list in the AFFINE mode with reference to a specific application scenario. Assuming that the motion information candidate list has a length of X, i.e. X multi-parameter models need to be added, X can be configured empirically, e.g. 5.
Referring to fig. 4F, G, C, a, B, and D are spatial neighboring blocks of the current block, and if at least one of a, B, and D is "available", the first "available" neighboring block X0 is obtained by scanning in the order of a, B, and D; otherwise, X0 is "absent". For example, at least one of a, B, and D is "available" to indicate that the motion information of at least one spatial neighboring block is available, and assuming that the neighboring block X0 is a, it indicates that the motion information of a is available.
If at least one of G and C is available, scanning according to the sequence of G and C to obtain a first available adjacent block X1; otherwise, X1 is "absent". At least one of G and C is 'available', which means that the motion information of at least one spatial neighboring block is available motion information, and if the neighboring block X1 is C, it means that the motion information of C is available motion information.
If F is available, the adjacent block F is marked as X2; otherwise, X2 is "absent".
If H is present, the notation H is X3; otherwise, X3 "does not exist", the H block is a collocated position, that is, a temporal neighboring block of the current block, if H "exists", the motion information of H is temporal motion information of the current block, and a derivation manner of the temporal motion information of the current block may refer to any embodiment in embodiments 1 to 6, which is not described herein again.
Illustratively, if X (X is X0, X1, or X2) "exists" and the L0 reference index of X is not equal to-1, then the L0 motion vector for X is MVX _ L0; otherwise MVX _ L0 is "not present". If X (X is X0, X1 or X2) "exists" and the L1 reference index of X is not equal to-1, then the L1 motion vector for X is MVX _ L1; otherwise MVX _ L1 is "not present". If X3 "exists" and the current picture is a B picture, deriving an L0 motion vector MVX3_ L0 and an L1 motion vector MVX3_ L1 in a manner of deriving the temporal motion information, wherein the reference indexes of L0 and L1 of X3 are both equal to 0; if X3 "exists" and the current picture is a P picture, deriving an L0 motion vector MVX3_ L0 in a manner of deriving the temporal motion information, the L0 reference index of X3 being equal to 0, the L1 reference index being equal to-1; otherwise MVX3_ L0 and MVX3_ L1 are "not present".
In summary, the motion information of X0, the motion information of X1, the motion information of X2, and the motion information of X3 can be obtained.
Illustratively, based on the motion information, a motion information candidate list of the AFFINE mode may be constructed, which in turn includes the following multi-parameter models: { CPMV0, CPMV1, CPMV2}, { CPMV0, CPMV1, CPMV3}, { CPMV0, CPMV2, CPMV3}, { CPMV1, CPMV2, CPMV3}, { CPMV0, CPMV1}, { CPMV0, CPMV2}, and CPMV0-CPMV3 are motion information generated for X0-X3 in sequence.
Illustratively, if the motion information candidate list does not reach five, zero motion information is padded.
Illustratively, the AFFINE mode prediction process includes, based on the motion information candidate list of the AFFINE mode:
calculating mv (v) of each small block by constructing a motion information candidate list of AFFINE mode using a multi-parameter model in the motion information candidate list x ，v y ) And motion compensation is performed on the small blocks. The calculation formula is as follows:
for a four-parameter model, the following formula is used to calculate mv (v) for each patch x ，v y )：
Figure GDA0003903178450000321
For the six-parameter model, the following formula is used to calculate mv (v) for each patch x ，v y )：
Figure GDA0003903178450000322
The affine motion compensation supports motion models of 4 parameters and 6 parameters, if the current block adopts bidirectional reference, sub-blocks mv are generated according to 8-by-8 small blocks and each sub-block is subjected to motion compensation, otherwise, the current block uses unidirectional motion information (forward reference or backward reference), sub-blocks mv are generated according to 4-by-4 or 8-by-8 small blocks, and each sub-block is subjected to motion compensation.
For example, the embodiments 1 to 10 may be implemented individually or in combination. For example, embodiment 1 and any one of embodiments 2 to 6 are implemented in combination, embodiment 7 and any one of embodiments 1 to 6 are implemented in combination, embodiment 8 and any one of embodiments 1 to 6 are implemented in combination, embodiment 9 and any one of embodiments 1 to 6 are implemented in combination, and embodiment 10 and any one of embodiments 1 to 6 are implemented in combination.
Of course, the above are only a few examples, and the combination manner between the embodiments is not limited.
Example 11: based on the same application concept as the method described above, an embodiment of the present application further provides an apparatus for constructing a motion information candidate list, which is applied to an encoding end or a decoding end, and is shown in fig. 6A, and is a structural diagram of the apparatus, including:
a determining module 611, configured to determine, if the current frame where the current block is located is a B frame, a time-domain neighboring block of the current block from a co-located frame of the current frame; and determining temporal motion information of the current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; wherein, if the forward motion information of the time domain neighboring block is not available and the backward motion information of the time domain neighboring block is available, the determining module 611 determines the backward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block; a constructing module 612, configured to construct a motion information candidate list of the current block based on the temporal motion information of the current block.
In one possible implementation, if the forward motion information of the time domain neighboring block is available and the backward motion information of the time domain neighboring block is not available, the determining module 611 determines the forward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block and determines the backward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block.
In a possible implementation manner, if the forward motion information of the time domain neighboring block is available and the backward motion information of the time domain neighboring block is available, the determining module 611 determines the forward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block and determines the backward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block.
In one possible implementation manner, if the forward motion information of the time domain neighboring block is available and the backward motion information of the time domain neighboring block is available, the determining module 611 determines the forward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block and determines the backward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block.
In one possible embodiment, if the forward motion information of the time-domain neighboring block is available and the backward motion information of the time-domain neighboring block is not available, the determining module 611 determines the forward motion information in the time-domain motion information according to the forward motion information of the time-domain neighboring block.
In a possible implementation manner, if the forward motion information of the time domain neighboring block is not available and the backward motion information of the time domain neighboring block is available, the determining module 611 determines the forward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block and determines the backward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block.
In one possible implementation, the motion information candidate list comprises a motion information candidate list of an angular weighted prediction mode; the building module 612 is specifically configured to: constructing a motion information candidate list of the current block based on available motion information to be added to the motion information candidate list; wherein the available motion information comprises temporal motion information of the current block; the available motion information further includes at least one of the following motion information: spatial domain motion information; HMVP motion information; and presetting motion information.
In one possible implementation, the motion information candidate list comprises a motion information candidate list of advanced motion information expression patterns; the building module 612 is specifically configured to: constructing a motion information candidate list of the current block based on available motion information to be added to the motion information candidate list; wherein the available motion information comprises temporal motion information of the current block; the available motion information further includes at least one of the following motion information: spatial domain motion information; and presetting motion information.
In a possible implementation, the motion information candidate list comprises a motion information candidate list of an affine motion compensation mode; the building module 612 is specifically configured to: acquiring a multi-parameter model based on available motion information; wherein the multi-parameter model comprises at least four parameters, and the available motion information comprises temporal motion information of the current block and motion information of spatial neighboring blocks of the current block; and constructing a motion information candidate list of the current block according to the multi-parameter model.
Based on the same application concept as the method described above, in terms of hardware level, a schematic diagram of a hardware architecture of a decoding-end device (which may also be referred to as a video decoder) provided in the embodiment of the present application may specifically be as shown in fig. 6B. The method comprises the following steps: a processor 621 and a machine-readable storage medium 622, wherein: the machine-readable storage medium 622 stores machine-executable instructions executable by the processor 621; the processor 621 is configured to execute machine executable instructions to implement the methods disclosed in the above examples of the present application. For example, the processor 621 is configured to execute machine-executable instructions to implement the following steps:
if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame;
determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block; and constructing a motion information candidate list of the current block based on the time domain motion information of the current block.
Based on the same application concept as the method described above, the hardware architecture diagram of the encoding end device (which may also be referred to as a video encoder) provided in the embodiment of the present application may specifically refer to fig. 6C from a hardware level. The method comprises the following steps: a processor 631 and a machine-readable storage medium 632, wherein: the machine-readable storage medium 632 stores machine-executable instructions executable by the processor 631; the processor 631 is configured to execute machine executable instructions to implement the methods disclosed in the above examples of the present application. For example, the processor 631 is configured to execute machine-executable instructions to perform the following steps:
if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame;
determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block; and constructing a motion information candidate list of the current block based on the time domain motion information of the current block.
Based on the same application concept as the method, embodiments of the present application further provide a machine-readable storage medium, where several computer instructions are stored, and when the computer instructions are executed by a processor, the method disclosed in the above examples of the present application, such as the method for constructing the motion information candidate list in the above embodiments, can be implemented.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices. For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. The present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims (15)

1. A method for constructing a motion information candidate list, the method comprising:
if the current frame where the current block is located is a B frame, determining a time domain adjacent block of the current block from a co-located frame of the current frame;
determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block and determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block;
constructing a motion information candidate list of the current block based on the time-domain motion information of the current block;
wherein the backward motion information of the time-domain neighboring block comprises a backward motion vector of the time-domain neighboring block and a backward reference frame index of the time-domain neighboring block, the backward reference frame index corresponds to a backward reference frame of the time-domain neighboring block, the backward motion information in the time-domain motion information comprises a backward motion vector of the current block and a backward reference frame index of the current block, and the backward reference frame index corresponds to a backward reference frame of the current block; and forward motion information in the temporal motion information includes a forward motion vector of the current block and a forward reference frame index of the current block, the forward reference frame index corresponding to a forward reference frame of the current block;
the determining backward motion information in the temporal motion information according to the backward motion information of the temporal neighboring block includes: based on the distance between the collocated frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the backward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information in the temporal motion information based on a backward motion vector of the current block and a backward reference frame index of the current block;
the determining forward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block includes: based on the distance between the co-located frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the backward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the forward motion vector of the current block; determining forward motion information in the temporal motion information based on a forward motion vector of the current block and a forward reference frame index of the current block.
2. The method of claim 1,
the determining temporal motion information for the current block based on motion information of temporal neighboring blocks comprises:
if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block and determining the backward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block;
wherein the forward motion information of the time domain neighboring block comprises forward motion vectors of the time domain neighboring block and forward reference frame indices of the time domain neighboring block, the forward reference frame indices corresponding to forward reference frames of the time domain neighboring block;
the determining forward motion information in the time domain motion information from forward motion information of the time domain neighboring blocks comprises: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the forward motion vector of the current block; determining forward motion information in the temporal motion information based on a forward motion vector of the current block and a forward reference frame index of the current block;
the determining backward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block includes: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information in the temporal motion information based on a backward motion vector of the current block and a backward reference frame index of the current block.
3. The method of claim 1,
the determining temporal motion information for the current block based on motion information of temporal neighboring blocks comprises:
if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is unavailable, determining the forward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block and determining the backward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block;
wherein the forward motion information of the time domain neighboring block comprises forward motion vectors of the time domain neighboring block and forward reference frame indices of the time domain neighboring block, the forward reference frame indices corresponding to forward reference frames of the time domain neighboring block;
the determining forward motion information in the time domain motion information according to the forward motion information of the time domain neighboring blocks comprises: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the forward motion vector of the current block; determining forward motion information in the temporal motion information based on a forward motion vector of the current block and a forward reference frame index of the current block;
the determining backward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block includes: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information in the temporal motion information based on a backward motion vector of the current block and a backward reference frame index of the current block.
4. The method of claim 1,
the determining temporal motion information for the current block based on motion information of temporal neighboring blocks comprises:
if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to an MVP mode;
wherein, the obtaining the time domain motion information of the current block according to the MVP mode includes: the L0 motion vector predicted value and the L1 motion vector predicted value obtained according to the MVP are respectively used as an L0 motion vector MvE0 and an L1 motion vector MvE1 of the current block, and an L0 reference index RefIdxL0 and an L1 reference index RefIdxL1 of the current block are both equal to 0.
5. The method according to any one of claims 1 to 4,
the determining the time-domain neighboring block of the current block from the co-located frame of the current frame comprises:
determining a time domain adjacent block corresponding to the preset position from a co-located frame of the current frame based on the preset position of the current block; wherein, the collocated frame is a corresponding frame with a reference frame index of 0 in the reference image queue 1 of the current frame.
6. The method according to any one of claims 1 to 4,
the motion information candidate list comprises a motion information candidate list of an angle weighted prediction mode;
the constructing a motion information candidate list of the current block based on the temporal motion information of the current block includes:
constructing a motion information candidate list of the current block based on available motion information to be added to the motion information candidate list;
wherein the available motion information comprises temporal motion information of the current block; the available motion information further includes at least one of the following motion information: spatial domain motion information; presetting motion information;
wherein the preset motion information includes: default motion information derived based on motion information already present in the motion information candidate list.
7. An apparatus for constructing a motion information candidate list, the apparatus comprising:
the determining module is used for determining a time domain adjacent block of the current block from a same-position frame of the current frame if the current frame where the current block is located is a B frame; determining temporal motion information of a current block based on motion information of temporal neighboring blocks, the temporal motion information including forward motion information and backward motion information; if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block and determining the backward motion information in the time domain motion information according to the backward motion information of the time domain adjacent block;
a construction module, configured to construct a motion information candidate list of the current block based on the temporal motion information of the current block;
wherein the backward motion information of the time-domain neighboring block comprises a backward motion vector of the time-domain neighboring block and a backward reference frame index of the time-domain neighboring block, the backward reference frame index corresponds to a backward reference frame of the time-domain neighboring block, the backward motion information in the time-domain motion information comprises a backward motion vector of the current block and a backward reference frame index of the current block, and the backward reference frame index corresponds to a backward reference frame of the current block; and forward motion information in the temporal motion information includes a forward motion vector of the current block and a forward reference frame index of the current block, the forward reference frame index corresponding to a forward reference frame of the current block;
the determining module is specifically configured to, when determining the backward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block: based on the distance between the collocated frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the backward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information in the temporal motion information based on the backward motion vector of the current block and a backward reference frame index of the current block;
the determining module is specifically configured to, when determining the forward motion information in the time domain motion information according to the backward motion information of the time domain neighboring block: based on the distance between the collocated frame and the backward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, performing stretching on the backward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the forward motion vector of the current block; determining forward motion information in the temporal motion information based on a forward motion vector of the current block and a forward reference frame index of the current block.
8. The apparatus of claim 7,
the determining module is specifically configured to, when determining the temporal motion information of the current block based on the motion information of the temporal neighboring block: if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is available, determining the forward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block and determining the backward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block;
wherein the forward motion information of the time domain neighboring block comprises forward motion vectors of the time domain neighboring block and forward reference frame indices of the time domain neighboring block, the forward reference frame indices corresponding to forward reference frames of the time domain neighboring block;
the determining module is specifically configured to, when determining the forward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the forward motion vector of the current block; determining forward motion information in the temporal motion information based on a forward motion vector of the current block and a forward reference frame index of the current block;
the determining module, when determining the backward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block, is specifically configured to: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information in the temporal motion information based on a backward motion vector of the current block and a backward reference frame index of the current block.
9. The apparatus of claim 7,
the determining module is specifically configured to, when determining the temporal motion information of the current block based on the motion information of the temporal neighboring block: if the forward motion information of the time domain adjacent block is available and the backward motion information of the time domain adjacent block is unavailable, determining the forward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block and determining the backward motion information in the time domain motion information according to the forward motion information of the time domain adjacent block;
wherein the forward motion information of the time domain neighboring block comprises forward motion vectors of the time domain neighboring block and forward reference frame indices of the time domain neighboring block, the forward reference frame indices corresponding to forward reference frames of the time domain neighboring block;
the determining module is specifically configured to, when determining the forward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the forward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the forward motion vector of the current block; determining forward motion information in the temporal motion information based on a forward motion vector of the current block and a forward reference frame index of the current block;
the determining module is specifically configured to, when determining the backward motion information in the time domain motion information according to the forward motion information of the time domain neighboring block: based on the distance between the co-located frame and the forward reference frame of the time domain adjacent block and the distance between the current frame and the backward reference frame of the current block, stretching the forward motion vector of the time domain adjacent block to obtain a stretched motion vector, and determining the stretched motion vector as the backward motion vector of the current block; determining backward motion information in the temporal motion information based on a backward motion vector of the current block and a backward reference frame index of the current block.
10. The apparatus of claim 7,
the determining module is specifically configured to, when determining the temporal motion information of the current block based on the motion information of the temporal neighboring block: if the forward motion information of the time domain adjacent block is unavailable and the backward motion information of the time domain adjacent block is unavailable, obtaining the time domain motion information of the current block according to an MVP mode; wherein, the determining module is specifically configured to, when obtaining the temporal motion information of the current block according to the MVP mode: the L0 motion vector predictor and the L1 motion vector predictor obtained according to MVP are respectively taken as the L0 motion vector MvE0 and the L1 motion vector MvE1 of the current block, and the L0 reference index RefIdxL0 and the L1 reference index RefIdxL1 of the current block are both equal to 0.
11. The apparatus according to any one of claims 7 to 10,
the determining module is specifically configured to, when determining the time-domain neighboring block of the current block from the co-located frame of the current frame: determining a time domain adjacent block corresponding to the preset position from a co-located frame of the current frame based on the preset position of the current block; wherein, the collocated frame is a corresponding frame with a reference frame index of 0 in the reference image queue 1 of the current frame.
12. The apparatus according to any one of claims 7 to 10,
the motion information candidate list comprises a motion information candidate list of an angle weighted prediction mode; the constructing module is specifically configured to, when constructing the motion information candidate list of the current block, based on the time-domain motion information of the current block:
constructing a motion information candidate list of the current block based on available motion information to be added to the motion information candidate list;
wherein the available motion information comprises temporal motion information of the current block; the available motion information further includes at least one of the following motion information: spatial domain motion information; presetting motion information; wherein the preset motion information includes: default motion information derived based on motion information already present in the motion information candidate list.
13. A decoding device, characterized by comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; wherein the content of the first and second substances,
the processor is configured to execute the machine executable instructions to implement the method of any of claims 1-6.
14. An encoding device, characterized by comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; wherein, the first and the second end of the pipe are connected with each other,
the processor is configured to execute the machine executable instructions to implement the method of any of claims 1-6.
15. A machine-readable storage medium having stored thereon machine-executable instructions executable by a processor; wherein the processor is configured to execute the machine executable instructions to implement the method of any of claims 1-6.
CN202111153145.1A 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list Active CN113873257B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111153145.1A CN113873257B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010637298.2A CN113891089B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list
CN202111153145.1A CN113873257B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202010637298.2A Division CN113891089B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list

Publications (2)

Publication Number Publication Date
CN113873257A CN113873257A (en) 2021-12-31
CN113873257B true CN113873257B (en) 2022-12-23

Family

ID=79000655

Family Applications (3)

Application Number Title Priority Date Filing Date
CN202111153145.1A Active CN113873257B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list
CN202010637298.2A Active CN113891089B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list
CN202111150918.0A Active CN114007078B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list

Family Applications After (2)

Application Number Title Priority Date Filing Date
CN202010637298.2A Active CN113891089B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list
CN202111150918.0A Active CN114007078B (en) 2020-07-03 2020-07-03 Method, device and equipment for constructing motion information candidate list

Country Status (1)

Country Link
CN (3) CN113873257B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024008011A1 (en) * 2022-07-05 2024-01-11 Beijing Bytedance Network Technology Co., Ltd. Method, apparatus, and medium for video processing

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3107290B1 (en) * 2014-03-14 2021-07-28 Samsung Electronics Co., Ltd. Method and device for configuring merge candidate list for decoding and encoding of interlayer video
US11025904B2 (en) * 2018-06-08 2021-06-01 Tencent America LLC Method and apparatus for temporal motion vector prediction
CN110933426B (en) * 2018-09-20 2022-03-01 杭州海康威视数字技术股份有限公司 Decoding and encoding method and device thereof
CN110958452B (en) * 2018-09-27 2023-11-03 华为技术有限公司 Video decoding method and video decoder
CN111263166B (en) * 2018-11-30 2022-10-11 华为技术有限公司 Video image prediction method and device
CN110225346A (en) * 2018-12-28 2019-09-10 杭州海康威视数字技术股份有限公司 A kind of decoding method and its equipment
CN116866605A (en) * 2019-01-03 2023-10-10 北京大学 Video processing method and device
CN111698506B (en) * 2019-03-11 2022-04-26 杭州海康威视数字技术股份有限公司 Motion information candidate list construction method, triangle prediction decoding method and device

Also Published As

Publication number Publication date
CN114007078B (en) 2022-12-23
CN114007078A (en) 2022-02-01
CN113891089B (en) 2022-12-23
CN113891089A (en) 2022-01-04
CN113873257A (en) 2021-12-31

Similar Documents

Publication Publication Date Title
CN111385569B (en) Coding and decoding method and equipment thereof
JP2020202590A (en) Dynamic image decoding device, dynamic image encoding device, dynamic image decoding method, and dynamic image encoding method
TWI616092B (en) Dynamic image predictive decoding device and dynamic image predictive decoding method
JP2018186569A (en) Image decoder
WO2010001917A1 (en) Image processing device and method
US20140205013A1 (en) Inter-prediction method and apparatus
WO2012172668A1 (en) Video encoding method and device, and video decoding method and device
CN111263144B (en) Motion information determination method and equipment
CN113873249B (en) Encoding and decoding method, device and equipment
CN112204962B (en) Image prediction method, apparatus and computer-readable storage medium
JP2012151576A (en) Image coding method, image coding device, image decoding method and image decoding device
CN113873257B (en) Method, device and equipment for constructing motion information candidate list
CN112291561B (en) HEVC maximum coding block motion vector calculation method, HEVC maximum coding block motion vector calculation device, HEVC maximum coding block motion vector chip and HEVC maximum coding block motion vector storage medium
CN113709499B (en) Encoding and decoding method, device and equipment
CN113810686B (en) Encoding and decoding method, device and equipment
WO2012090425A1 (en) Moving image encoding device, moving image encoding method, and moving image encoding program, as well as moving image decoding device, moving image decoding method, and moving image decoding program
KR102380722B1 (en) Method for inter prediction and apparatus thereof
JP5946980B1 (en) Image decoding method
JP6181242B2 (en) Image decoding method
JP5951915B2 (en) Image decoding method
JP5750191B2 (en) Image decoding method
JP2012138838A (en) Moving image decoding apparatus, moving image decoding method and moving image decoding program
CN116684578A (en) Affine model optimization based on control point motion vectors
JP2006157214A (en) Moving picture coding apparatus
JP2015111925A (en) Image coding method, image coding device, image decoding method and image decoding device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40064078

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant