CN110662033B - Decoding and encoding method and device thereof - Google Patents

Decoding and encoding method and device thereof Download PDF

Info

Publication number
CN110662033B
CN110662033B CN201810692041.XA CN201810692041A CN110662033B CN 110662033 B CN110662033 B CN 110662033B CN 201810692041 A CN201810692041 A CN 201810692041A CN 110662033 B CN110662033 B CN 110662033B
Authority
CN
China
Prior art keywords
image block
motion vector
template
candidate
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810692041.XA
Other languages
Chinese (zh)
Other versions
CN110662033A (en
Inventor
陈方栋
王莉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201810692041.XA priority Critical patent/CN110662033B/en
Priority to PCT/CN2019/093574 priority patent/WO2020001591A1/en
Publication of CN110662033A publication Critical patent/CN110662033A/en
Application granted granted Critical
Publication of CN110662033B publication Critical patent/CN110662033B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/573—Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The application provides a decoding and encoding method and a device thereof, wherein the method comprises the following steps: acquiring motion information of candidate image blocks of a current image block; acquiring a template of the current image block according to the motion information of the candidate image block; decoding an encoded bitstream according to the template of the current image block. According to the technical scheme, the template of the current image block can be obtained quickly, the problems of low decoding efficiency, large decoding time delay and the like are solved, the decoding efficiency can be improved, the decoding time delay is reduced, and the encoding and decoding performance is improved.

Description

Decoding and encoding method and device thereof
Technical Field
The present application relates to the field of video encoding and decoding technologies, and in particular, to a decoding method, an encoding method, and an apparatus thereof.
Background
In order to achieve the purpose of saving space, video images are transmitted after being coded, and the complete video coding method can comprise the processes of prediction, transformation, quantization, entropy coding, filtering and the like. The predictive coding comprises intra-frame coding and inter-frame coding, wherein the inter-frame coding is to use the correlation of a video time domain and use the pixels adjacent to a coded image to predict the pixels of a current image so as to achieve the aim of effectively removing the video time domain redundancy.
In the video coding technique, the coding process is performed on a block-by-block basis, when the current block is coded, the reconstruction information of the surrounding coded blocks is available, and the Template (Template) refers to the decoding information of the fixed shape around the current block (adjacent area in the time domain or the space domain). The templates are identical at the encoding end and the decoding end, so that some operations performed by the templates at the encoding end can obtain completely consistent results at the decoding end, that is, information derived by the encoding end based on the templates can be recovered at the decoding end without loss, without transferring additional information, thereby further reducing the number of transmission bits.
However, the conventional template generation method has the problems of low decoding efficiency, large decoding delay and the like.
Disclosure of Invention
The application provides a decoding and encoding method and equipment thereof, which solve the problems of low decoding efficiency, large decoding time delay and the like, can improve the decoding efficiency, reduce the decoding time delay and improve the encoding and decoding performance.
The application provides a decoding method, which is applied to a decoding end and comprises the following steps:
acquiring motion information of candidate image blocks of a current image block;
acquiring a template of the current image block according to the motion information of the candidate image block;
decoding an encoded bitstream according to the template of the current image block.
The application provides a coding method, which is applied to a coding end, and the method comprises the following steps:
acquiring motion information of candidate image blocks of a current image block;
acquiring a template of the current image block according to the motion information of the candidate image block;
coding a bit stream according to the template of the current image block to obtain a coded bit stream;
and sending the coded bit stream to a decoding end.
The application provides a decoding end device, decoding end device includes: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute machine-executable instructions to perform the method steps described above.
The application provides a coding end device, the coding end device includes: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute machine-executable instructions to perform the method steps described above.
According to the technical scheme, in the embodiment of the application, the template of the current image block can be obtained according to the motion information of the candidate image block of the current image block, and the coded bit stream is decoded according to the template of the current image block. By the method, the template of the current image block can be quickly obtained, the problems of low decoding efficiency, large decoding time delay and the like are solved, the decoding efficiency can be improved, the decoding time delay is reduced, and the encoding and decoding performance is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings of the embodiments of the present application.
FIG. 1 is a flow chart of an encoding method in one embodiment of the present application;
FIGS. 2A-2O are schematic diagrams of a template of a target image block in one embodiment of the present application;
FIGS. 3A-3D are flow diagrams of an encoding method in another embodiment of the present application;
FIG. 4 is a flow chart of a decoding method in one embodiment of the present application;
FIG. 5 is a flow chart of a decoding method in another embodiment of the present application;
fig. 6 is a block diagram of a decoding apparatus according to an embodiment of the present application;
fig. 7 is a block diagram of an encoding device according to an embodiment of the present application;
fig. 8 is a hardware configuration diagram of a decoding-side device according to an embodiment of the present application;
fig. 9 is a hardware configuration diagram of an encoding-side device according to an embodiment of the present application.
Detailed Description
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Depending on the context, moreover, the word "if" as used may be interpreted as "at … …" or "when … …" or "in response to a determination".
The embodiment of the application provides a decoding and encoding method, which can relate to the following concepts:
motion Vector (MV): in inter-frame coding, a motion vector is used to represent a relative displacement between a current image block of a current frame video image and a reference image block of a reference frame video image, for example, there is a strong temporal correlation between video image a of the current frame and video image B of the reference frame, when transmitting image block a1 (current image block) of video image a, a motion search may be performed in video image B to find image block B1 (reference image block) that best matches image block a1, and determine a relative displacement between image block a1 and image block B1, which is also the motion vector of image block a 1. Wherein each divided image block has a corresponding motion vector transmitted to a decoding side, and considerable bits are consumed if the motion vector of each image block is independently encoded and transmitted, especially divided into a large number of image blocks of small size. In order to reduce the bit number used for encoding the motion vector, the spatial correlation between adjacent image blocks can be utilized, the motion vector of the current image block to be encoded is predicted according to the motion vector of the adjacent encoded image block, and then the prediction difference is encoded, so that the bit number representing the motion vector can be effectively reduced.
Motion Information (Motion Information): since the motion vector indicates the position offset of the current image block from a certain reference image block, in order to accurately acquire information pointing to the image block, index information of the reference frame image is required in addition to the motion vector to indicate which reference frame image is used. In the video coding technology, for a current frame image, a reference frame image list may be generally established, and the reference frame image index information indicates that the current image block adopts the several reference frame images in the reference frame image list. In the video encoding technology, motion-related encoding information such as a motion vector and a reference frame index may be collectively referred to as motion information.
Template (Template): in video coding techniques, the coding process is performed on an image block-by-image block basis, and when the current image block is coded, the reconstruction information of the surrounding coded image blocks is available. The template refers to the decoding information of a fixed shape around the current image block (adjacent region in the time domain or the space domain). The templates are identical at the encoding end and the decoding end, so that some operations performed by the templates at the encoding end can obtain completely consistent results at the decoding end, that is, information derived by the encoding end based on the templates can be recovered at the decoding end without loss, without transferring additional information, thereby further reducing the number of transmission bits.
Rate-Distortion principle (Rate-Distortion Optimized): there are two major indicators for evaluating coding efficiency: code rate and Peak Signal to Noise Ratio (PSNR), the smaller the bit stream, the larger the compression rate, and the larger the PSNR, the better the reconstructed image quality, and in the mode selection, the discrimination formula is essentially the comprehensive evaluation of the two. For example, the cost for a mode: j (mode) ═ D + λ R, where D denotes Distortion, which can be generally measured using SSE index, SSE being the sum of the mean square of the differences between the reconstructed image block and the source image; and λ is a lagrange multiplier, and R is the actual number of bits required for encoding the image block in the mode, including the sum of bits required for encoding mode information, motion information, residual errors and the like.
Intra and inter prediction (intra and inter) techniques: the intra-frame prediction refers to performing predictive coding by using reconstructed pixel values of spatial adjacent image blocks of a current image block (i.e., the image in the same frame as the current image block), and the inter-frame prediction refers to performing predictive coding by using reconstructed pixel values of temporal adjacent image blocks of the current image block (i.e., the image in a different frame from the current image block).
The decoding method and the encoding method will be described in detail below with reference to specific embodiments.
The first embodiment is as follows:
referring to fig. 1, a flow chart of an encoding method is shown, and the method may include the following steps:
step 101, a coding end acquires motion information of a candidate image block of a current image block.
The candidate image blocks of the current image block may include, but are not limited to: a spatial domain candidate image block of the current image block; or, a time domain candidate image block of the current image block; the candidate image block is not limited.
When the current image block is encoded, since the candidate image block of the current image block has already been encoded, that is, the motion information of the candidate image block is known, the encoding end may directly obtain the motion information of the candidate image block, such as a motion vector and a reference frame index of the candidate image block, which is not limited thereto.
And 102, the encoding end acquires a template of the current image block according to the motion information of the candidate image block.
The processing procedure in step 102 may refer to the following embodiments, which are not described herein again.
And 103, the encoding end encodes the bit stream according to the template of the current image block to obtain an encoded bit stream. Wherein the coded bitstream may be a coded bitstream for the current image block.
The encoding of the bitstream according to the template of the current image block to obtain the encoded bitstream may include, but is not limited to: acquiring original mode information of a current image block; acquiring target mode information of the current image block according to the original mode information based on the template of the current image block; and coding the bit stream according to the target mode information to obtain a coded bit stream. The original mode information is an original motion vector, and the target mode information is a target motion vector; or the original mode information is an original motion vector and an original reference frame, and the target mode information is a target motion vector and a target reference frame; or the original mode information is an original prediction image block, and the target mode information is a target prediction image block. Of course, the above is merely an example, and no limitation is made thereto.
And step 104, the encoding end sends the encoded bit stream to the decoding end.
When the encoding end encodes the bit stream according to the target mode information, the encoding end sends the encoded bit stream carrying first indication information to the decoding end, the first indication information is used for indicating a template based on the current image block, and the target mode information of the current image block is obtained according to the original mode information. When the encoding end encodes the bit stream according to the original mode information, the encoding end sends the encoding bit stream carrying second indication information to the decoding end, and the second indication information is used for indicating that the encoding bit stream is decoded based on the original mode information.
In practical application, the first indication information or the second indication information may also be notified in an implicit manner, that is, the first indication information or the second indication information is not carried in the coded bitstream. Specifically, the encoding end and the decoding end may also negotiate a decision policy or define a policy in a standard, and store the policy in the encoding end and the decoding end, where the decision policy may be default first policy information, where the first policy information is used to indicate a template based on a current image block, and obtain target mode information of the current image block according to original mode information; or, default to second policy information for indicating decoding of the encoded bitstream based on the original mode information; or, the third policy information is defaulted, and the third policy information is the same policy information as that adopted by the adjacent image block of the current image block.
On this basis, when the decision policy is the default first policy information and the encoding end encodes the bitstream according to the target mode information, the encoded bitstream may not carry the first indication information and the second indication information. When the decision policy is the default second policy information and the encoding end encodes the bitstream according to the original mode information, the encoded bitstream may not carry the first indication information and the second indication information. When the decision policy is the default third policy information and the adjacent image block adopts the first policy information, the coding end may not carry the first indication information and the second indication information when coding the bitstream according to the target mode information; when the adjacent image block uses the second policy information, the encoding end may not carry the first indication information and the second indication information when encoding the bitstream according to the original mode information.
According to the technical scheme, in the embodiment of the application, the template of the current image block can be obtained according to the motion information of the candidate image block of the current image block, and the bit stream is encoded according to the template of the current image block to obtain the encoded bit stream. The method can quickly obtain the template of the current image block, solves the problems of low coding efficiency, large coding delay and the like, can improve the coding efficiency, reduce the coding delay and improve the coding performance.
Example two:
in the process of video coding, each image block is coded one by one, and when the current image block is coded, if the adjacent image blocks around the current image block are completely reconstructed, the current image block can be reconstructed by using the decoded information of the adjacent image blocks, so that the template of the current image block can be obtained by using the decoded information of the adjacent image blocks of the current image block. The decoded information may include, but is not limited to: reconstruction information of neighboring image blocks and/or prediction information of neighboring image blocks. The reconstruction information may include, but is not limited to, luminance values, chrominance values, and the like; the prediction information may be an intermediate value from which the reconstruction information can be obtained, and for example, if the luminance value can be obtained using the intermediate value a, the intermediate value a is the prediction information, and this prediction information is not limited.
However, if the decoded information is reconstruction information, the generation of the template of the current image block needs to wait for the decoding reconstruction stage, which greatly reduces the decoding efficiency and brings decoding delay; and the decoding information is prediction information, the generation of the template of the current image block also needs to wait for the decoding reconstruction stage, so that the decoding efficiency is greatly reduced, and the decoding time delay is brought. Therefore, the above method may cause the parallelism of decoding to be greatly affected.
In view of the above findings, in this embodiment, a better template generating manner is provided, and unlike a manner in which a template is generated by using reconstruction information and prediction information, a method for obtaining the template of the current image block according to motion information (such as a motion vector and a reference frame index) of a candidate image block of the current image block and obtaining the template of the current image block according to the motion information of the candidate image block may be applied to both the encoding side and the decoding side.
The obtaining the template of the current image block according to the motion information of the candidate image block may include: when the motion information comprises a motion vector and a reference frame index of the candidate image block, determining a reference frame image corresponding to the candidate image block according to the reference frame index; and acquiring a reference image block corresponding to the candidate image block from the reference frame image according to the motion vector, and acquiring a template of the current image block according to the reference image block.
For example, referring to FIG. 2A, assume tile A1 is the current tile and tile A2 and tile A3 are candidate tiles for tile A1. For tile a2, if the reference frame index of tile a2 is the index of video image B, it may be determined that the reference frame image corresponding to tile a2 is video image B according to the reference frame index; then, an image block B2 corresponding to the image block A2 is selected from the video image B, namely the position of the image block B2 in the video image B is the same as the position of the image block A2 in the video image A; then, the image block B2 may be moved according to the motion vector of the image block a2, for example, the image block B2 is moved by using the motion vector (3, 3), so as to obtain the image block B2 '(e.g., moving 3 pixels rightward and moving 3 pixels upward) corresponding to the image block B2, and the image block B2' is a reference image block corresponding to the image block a 2. Likewise, the reference image block corresponding to image block a3 may be determined to be image block B3'. Further, a template for tile A2 may be determined from tile B2 'and tile B3', as illustrated in FIG. 2A.
In one example, the candidate image blocks may include M first candidate image blocks and N second candidate image blocks, where M is a natural number greater than or equal to 1 and N is a natural number greater than or equal to 0, or M is a natural number greater than or equal to 0 and N is a natural number greater than or equal to 1. The first candidate image block is a candidate image block on the upper side of the current image block, and the second candidate image block is a candidate image block on the left side of the current image block.
Obtaining the template of the current image block according to the motion information of the candidate image block may include, but is not limited to: determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks; and determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks. Then, determining the first template as the template of the current image block; or, determining the second template as the template of the current image block; or determining the first template and the second template as the template of the current image block after splicing.
For example, if M is a natural number greater than or equal to 1 and N is 0, the first template may be determined according to the motion vector prediction modes and the motion information of the M first candidate image blocks, and the first template may be determined as the template of the current image block. If N is a natural number greater than or equal to 1 and M is 0, the second template may be determined according to the motion vector prediction modes and the motion information of the N second candidate image blocks, and the second template may be determined as the template of the current image block. If M is a natural number greater than or equal to 1 and N is a natural number greater than or equal to 1, determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks, determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks, and determining the first template as the template of the current image block; or determining the second template as the template of the current image block; or determining the first template and the second template as the template of the current image block after splicing.
The first candidate image block comprises an adjacent image block and/or a secondary adjacent image block on the upper side of the current image block; the prediction mode of the adjacent image block is an inter mode or an intra mode; the prediction mode of the secondary adjacent image block is an inter mode. For example, the first candidate image block may include at least one neighboring image block whose prediction mode is an inter mode, e.g., all neighboring image blocks on an upper side of the current image block, or a first neighboring image block on an upper side of the current image block, or any one or more neighboring image blocks on an upper side of the current image block. Furthermore, when the adjacent image blocks on the upper side of the current image block are both in intra mode, the first candidate image block may further include at least one secondary adjacent image block of which the prediction mode is inter mode, for example, all secondary adjacent image blocks on the upper side of the current image block, or the first secondary adjacent image block on the upper side of the current image block, or any one or more secondary adjacent image blocks on the upper side of the current image block. Furthermore, when there are neighboring image blocks in intra mode on the top side of the current image block, the first candidate image block may further include neighboring image blocks in intra mode, for example, a neighboring image block in first intra mode on the top side of the current image block, neighboring image blocks in all intra modes on the top side of the current image block, and the like. Of course, the above is only an example of the first candidate image block, and the method is not limited thereto.
The second candidate image block comprises an adjacent image block on the left side of the current image block and/or a secondary adjacent image block; the prediction mode of the adjacent image block is an inter mode or an intra mode; the prediction mode of the secondary adjacent image block is an inter mode. For example, the second candidate image block may include at least one neighboring image block whose prediction mode is an inter mode, e.g., all neighboring image blocks on the left side of the current image block, or a first neighboring image block on the left side of the current image block, or any one or more neighboring image blocks on the left side of the current image block. Furthermore, when the adjacent image blocks on the left side of the current image block are all in intra mode, the second candidate image block may further include at least one secondary adjacent image block of which the prediction mode is in inter mode, for example, all secondary adjacent image blocks on the left side of the current image block, or the first secondary adjacent image block on the left side of the current image block, or any one or more secondary adjacent image blocks on the left side of the current image block. Furthermore, when there is an intra-mode neighboring image block on the left side of the current image block, the first candidate image block may further include intra-mode neighboring image blocks, for example, a first intra-mode neighboring image block on the left side of the current image block, all intra-mode neighboring image blocks on the left side of the current image block, and the like. Of course, the above is only an example of the second candidate image block, and the method is not limited thereto.
Wherein, the neighboring image blocks of the current image block include but are not limited to: spatial adjacent image blocks of the current image block (i.e. adjacent image blocks in the same frame of video image); or temporally neighboring image blocks of the current image block (i.e., neighboring image blocks in a different frame of the video image). The secondary neighboring image blocks of the current image block include, but are not limited to: the spatial domain secondary adjacent image blocks of the current image block (namely the secondary adjacent image blocks in the same frame of video image); or a temporal secondary neighboring image block of the current image block (i.e., a secondary neighboring image block in a different frame of video image).
In an example, when M is greater than 1, the first template may include M sub-templates or P sub-templates, and the M sub-templates or the P sub-templates are spliced, where P may be the number of first candidate image blocks in the inter mode, and P is less than or equal to M. For example, if the M first candidate image blocks are all candidate image blocks in the inter mode, the first template may include M sub-templates, and the M sub-templates are spliced together. For another example, if the M first candidate image blocks include candidate image blocks in P inter modes and include candidate image blocks in M-P intra modes, the first template may include M sub-templates (i.e., one sub-template for each candidate image block) and is formed by splicing M sub-templates, or the first template may include P sub-templates (i.e., P sub-templates for the candidate image blocks in P inter modes) and is formed by splicing P sub-templates.
Furthermore, when M is equal to 1, the first template may include a first sub-template, which may be determined according to the motion vector prediction mode and the motion information of the first candidate image block on the upper side of the current image block; alternatively, the first sub-template may be determined according to the motion vector prediction mode and the motion information of any candidate image block on the upper side of the current image block. Since the first candidate image block includes at least one adjacent image block or a sub-adjacent image block whose prediction mode is the inter mode, when M is equal to 1, the first template includes a first sub-template corresponding to the adjacent image block or the sub-adjacent image block in the inter mode.
In one example, the motion information may include a motion vector and a reference frame index of the first candidate image block, based on which the first template is determined according to the motion vector prediction modes and the motion information of the M first candidate image blocks, which may include but is not limited to:
in the first case, for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter-frame mode, a reference frame image corresponding to the ith candidate image block is determined according to a reference frame index; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block; then, image blocks with the sizes of the first transverse length and the first longitudinal length may be obtained as the ith sub-template included in the first template according to the determined reference image block.
And secondly, for the ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be the intra-frame mode, filling the ith candidate image block according to a default value (such as a default pixel value, which may be an empirically preconfigured brightness value), and acquiring the image blocks with the sizes of the first transverse length and the first longitudinal length as the ith sub-template included in the first template based on the image block filled with the default value.
Determining a reference frame image corresponding to the ith candidate image block according to a reference frame index corresponding to the ith candidate image block when the motion vector prediction mode of the ith candidate image block is determined to be the intra-frame mode; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with (equal to or approximately equal to) the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a first transverse length and a first longitudinal length as an ith sub-template included in the first template; and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the image block adjacent to the ith candidate image block.
The first lateral length and the lateral length of the first candidate image block satisfy a first proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc., which is not limited), or satisfy a second proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc.) with the lateral length of the current image block, or are equal to a first preset length (configured empirically).
The first vertical length and the vertical length of the first candidate image block satisfy a third proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc.), or satisfy a fourth proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc.) with the vertical length of the current image block, or equal to a second preset length (i.e., a length configured empirically).
The first proportional relationship, the second proportional relationship, the third proportional relationship and the fourth proportional relationship may be the same or different. The first preset length and the second preset length may be set to be the same or different.
In an example, when N is greater than 1, the second template may include N sub-templates or R sub-templates, and the N sub-templates or R sub-templates are spliced, where R may be the number of second candidate image blocks in the inter mode, and R is less than or equal to N. For example, if the N second candidate image blocks are all candidate image blocks in the inter mode, the second template may include N sub-templates, and the N sub-templates are spliced together. For another example, if the N second candidate image blocks include candidate image blocks in R inter modes and include candidate image blocks in N-R intra modes, the second template may include N sub-templates (i.e., one sub-template for each candidate image block) and is formed by splicing the N sub-templates, or the second template may include R sub-templates (i.e., R sub-templates for the candidate image blocks in R inter modes) and is formed by splicing the R sub-templates.
Furthermore, when N is equal to 1, the second template may include a second sub-template, which may be determined according to the motion vector prediction mode and the motion information of the first candidate image block on the left side of the current image block; alternatively, the second sub-template may be determined according to the motion vector prediction mode and the motion information of any candidate image block on the left side of the current image block. The second candidate image block includes at least one adjacent image block or a second adjacent image block of which the prediction mode is the inter mode, and therefore when N is equal to 1, the second template includes a second sub-template corresponding to the adjacent image block or the second adjacent image block of the inter mode.
In one example, the motion information may include a motion vector and a reference frame index of the second candidate image block, based on which the second template is determined according to the motion vector prediction modes and the motion information of the N second candidate image blocks, which may include but is not limited to:
in case one, for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter-frame mode, determining a reference frame image corresponding to the ith candidate image block according to a reference frame index; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block; then, image blocks having the sizes of the second transverse length and the second longitudinal length may be obtained as the ith sub-template included in the second template according to the determined reference image block.
And in a second case, for an ith candidate image block in the N second candidate image blocks, when it is determined that the motion vector prediction mode of the ith candidate image block is the intra-frame mode, filling the ith candidate image block according to a default value (for example, a default pixel value, which may be a luminance value configured in advance according to experience), and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as an ith sub-template included in the second template based on the image block filled with the default value.
Determining a reference frame image corresponding to the ith candidate image block according to a reference frame index corresponding to the ith candidate image block when the motion vector prediction mode of the ith candidate image block is determined to be the intra-frame mode; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with (equal to or approximately equal to) the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a second transverse length and a second longitudinal length as an ith sub-template included in the first template; and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the image block adjacent to the ith candidate image block.
The second horizontal length and the horizontal length of the second candidate image block satisfy a fifth proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc., which is not limited), or satisfy a sixth proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc.) with the horizontal length of the current image block, or are equal to a third preset length (configured empirically).
The second vertical length and the vertical length of the second candidate image block satisfy a seventh proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc.), or satisfy an eighth proportional relationship (e.g., 1: 1, 1: 2, 2: 1, etc.) with the vertical length of the current image block, or are equal to a fourth preset length (i.e., a length configured empirically).
The fifth proportional relation, the sixth proportional relation, the seventh proportional relation and the eighth proportional relation may be the same or different. The third preset length and the fourth preset length may be set to be the same or different.
In an example, obtaining the template of the current image block according to the motion information of the candidate image block may further include, but is not limited to: when the current image block corresponds to a plurality of pieces of motion information, a template corresponding to the motion information is obtained according to each piece of motion information, and the obtaining mode of each template refers to the above embodiment. Then, a weight parameter corresponding to each piece of motion information is obtained, and a template of the current image block is obtained according to the weight parameter corresponding to each piece of motion information and the template corresponding to the motion information. For example, the template of the current image block is obtained based on a weighted average method based on the weight parameter corresponding to each piece of motion information and the template corresponding to the piece of motion information.
For example, the current image block corresponds to motion information a and motion information B, the template a corresponding to the motion information a is obtained by using the above embodiment, and the template B corresponding to the motion information B is obtained by using the above embodiment.
Then, the weight parameter 1 of the motion information a and the weight parameter 2 of the motion information B may be obtained, so that the template of the current image block may be (template a × weight parameter 1+ template B × weight parameter 2)/2.
The template of the current image block is described in detail below with reference to several specific cases.
The first condition is as follows: referring to fig. 2B, the candidate image blocks may include all inter-mode neighboring image blocks on the upper side of the current image block and all inter-mode neighboring image blocks on the left side of the current image block.
For the current block a1, if there are neighboring blocks of the inter mode, such as block A3 and block a4, on the left side, the block A3 and block a4 of the inter mode may be determined as candidate blocks for the current block a 1. Similarly, if there are neighboring tiles of the inter mode, such as tile a2, on the upper side, the tile a2 of the inter mode may be determined as a candidate tile for the current tile a 1.
If there is no available image block on the left side, or if the available image blocks are all in intra mode although there is an available image block on the left side, it may indicate that there is no candidate image block on the left side of the current image block a 1. Similarly, if there is no available image block on the upper side, or if the available image blocks are all in intra mode although there is an available image block on the upper side, it may indicate that there is no candidate image block on the upper side of the current image block a 1. If neither the left side nor the top side has a candidate image block, it may indicate that the current image block a1 has no candidate image block.
In addition, if the current image block a1 has no candidate image block, the technical solution of the present embodiment is not adopted, but a conventional manner is adopted, which is not described again. If there are candidate image blocks in the current image block a1, such as a left candidate image block and/or an upper candidate image block, the technical solution of the present embodiment is adopted.
In an example, after determining that the candidate image block is the image block a2, the image block A3, and the image block a4, the template of the current image block a1 may be obtained according to the motion information of the image block a2, the motion information of the image block A3, and the motion information of the image block a 4. For example, for the tile a2, a reference frame image corresponding to the tile a2 may be determined according to the reference frame index, the tile B2 corresponding to the tile a2 is selected from the reference frame image, and the tile B2 is moved according to the motion vector of the tile a2 to obtain a reference tile B2 ' corresponding to the tile a2, and similarly, a reference tile B3 ' corresponding to the tile A3 and a reference tile B4 ' corresponding to the tile a4 may be obtained, as shown in fig. 2C. The template for the current tile A1 may then be obtained from reference tile B2 ', reference tile B3 ', and reference tile B4 '.
In an example, assuming that the lateral length of the upper template of the current image block a1 is W, the longitudinal length is S, the value of W may be configured empirically, the value of S may be configured empirically, and the values of W and S are not limited. For example, W may be the lateral length of the current image block A1, the lateral length of the candidate image block A2, 2 times the lateral length of the current image block A1, or the like, S may be the longitudinal length of the candidate image block A2, 1/3 the longitudinal length of the candidate image block A2, or the like. On the basis, refer to fig. 2D, which is a schematic diagram of a template corresponding to the reference image block B2'. In fig. 2D, W is taken as the lateral length of candidate image block a2 as an example, i.e. W is the lateral length of reference image block B2'; s is taken as 1/3, the longitudinal length of candidate image block a2, i.e., S is 1/3, the longitudinal length of reference image block B2'.
Assuming that the transverse length of the left template of the current image block a1 is R, the longitudinal length is H, the value of R may be configured according to experience, the value of H may be configured according to experience, and the values of R and H are not limited. For example, H may be the longitudinal length of the current image block A1, the longitudinal length of the candidate image block A3, R may be the lateral length of the candidate image block A3, 1/3 which is the lateral length of the candidate image block A3, or the like. On this basis, referring to fig. 2D, a template diagram corresponding to the reference image block B3' is shown, where H is taken as the longitudinal length of the candidate image block A3, and R is taken as the 1/3 of the transverse length of the candidate image block A3.
Similarly, the template page corresponding to the reference image block B4' can be seen in fig. 2D, and is not described herein again.
In one example, assuming that there are M candidate image blocks with different modes on the top side of the current image block, for the ith candidate image block, its lateral length is assumed to be wiThen, the prediction mode of the candidate image block is determined.
In the case of the intra mode, the sub-template is not generated any more, or is filled in by a default value (for example, a default pixel value, which may be a luminance value configured in advance according to experience) as the ith sub-template of the upper template.
If the image is in the inter-frame mode, motion information (such as a motion vector and a reference frame index) of the ith candidate image block is acquired, and a horizontal length w is generated based on the motion vector and the reference frame indexiThe template with the longitudinal length S is used as the ith sub-template of the upper template. Specifically, if the motion vector is MV and the reference frame index is idx, find the horizontal length w with the relative position offset MV in the idx-th reference picture of the current frameiAnd the rectangular block with the longitudinal length of S is used as the ith sub-template of the upper template.
Assuming that there are N candidate image blocks with different modes on the left side of the current image block, for the ith candidate image block, assuming that the longitudinal length is hiAnd if the transverse length is R, judging the prediction mode of the candidate image block.
In the intra mode, no sub-template is generated, or the left side template is filled with default values (e.g., default pixel values, which may be empirically pre-configured luminance values) as the ith sub-template.
If the image is in the inter-frame mode, motion information (such as a motion vector, a reference frame index and the like) of the ith candidate image block is acquired, and a horizontal length R and a vertical length h are generated based on the motion vector and the reference frame indexiAs the ith sub-template of the left template. Specifically, if the motion vector is MV and the reference frame index is idx, find the horizontal length R and the vertical length h with the relative position offset of MV in the idx reference image of the current frameiAs the ith sub-template of the left template.
Further, the upper side template can be formed by splicing all the sub-templates on the upper side, the left side template can be formed by splicing all the sub-templates on the left side, and the upper side template and the left side template are spliced into the template of the current image block.
Case two: referring to fig. 2E, the candidate image blocks may include a first inter-mode adjacent image block on the upper side of the current image block and a first inter-mode adjacent image block on the left side of the current image block.
For the current tile A1, if the first tile A3 on the left is inter mode, then tile A3 may be determined to be a candidate tile for the current tile A1. If the first tile A2 on the upper side is in inter mode, then tile A2 may be determined to be a candidate tile for the current tile A1.
Wherein if no image block is available on the left side, or if the first image block on the left side is in intra mode, then there is no candidate image block on the left side of the current image block a 1. If there is no available image block on the upper side, or the first image block on the upper side is in intra mode, there is no candidate image block on the upper side of the current image block a 1. If neither the left nor the top side has a candidate image block, the current image block a1 has no candidate image block.
Further, if the current image block a1 has no candidate image block, the technical solution of the present embodiment is not adopted, but a conventional method is adopted, which is not described again. If there are candidate image blocks in the current image block a1, such as a left candidate image block and/or an upper candidate image block, the technical solution of the present embodiment is adopted.
After determining that the candidate image blocks are the image block a2 and the image block A3, the template of the current image block a1 may be obtained according to the motion information of the image block a2 and the motion information of the image block A3. For example, the reference frame image corresponding to the image block a2 may be determined according to the reference frame index, the image block B2 corresponding to the image block a2 is selected from the reference frame image, the image block B2 is moved according to the adjacent motion vector of the image block a2 to obtain the reference image block B2 'corresponding to the image block a2, and similarly, the reference image block B3' corresponding to the image block A3 may be obtained, and the template may be obtained according to the reference image block B2 'and the reference image block B3'.
In an example, assuming that the lateral length of the upper template of the current image block a1 is W, the longitudinal length is S, the value of W may be configured empirically, the value of S may be configured empirically, and the values of W and S are not limited. For example, W may be the lateral length of the current image block A1, the lateral length of the candidate image block A2, S may be the longitudinal length of the candidate image block A2, 1/3 which is the longitudinal length of the candidate image block A2, or the like. Referring to fig. 2F, it is a schematic diagram of a template corresponding to the reference image block B2'.
Assuming that the transverse length of the left template of the current image block a1 is R, the longitudinal length is H, the value of R may be configured according to experience, the value of H may be configured according to experience, and the values of R and H are not limited. For example, H may be the longitudinal length of the current image block A1, the longitudinal length of the candidate image block A3, or the like, and R may be the lateral length of the candidate image block A3, 1/3 which is the lateral length of the candidate image block A3, or the like. On the basis, refer to fig. 2F, which is a schematic diagram of a template corresponding to the reference image block B3'.
In one example, assuming that there are M candidate image blocks with different modes on the upper side of the current image block, the prediction mode of the candidate image block is determined for the first candidate image block on the upper side, assuming that the lateral length is w. In the intra mode, no template is generated, or the template is filled in with default values (e.g., default pixel values, which may be empirically pre-configured luminance values) as the upper template. If the image block is in the inter-frame mode, motion information (such as a motion vector, a reference frame index and the like) of the candidate image block is acquired, and a template with a horizontal length w and a vertical length S is generated as an upper template based on the motion vector and the reference frame index.
And (2) assuming that N candidate image blocks with different modes exist on the left side of the current image block, and for the first candidate image block on the left side, assuming that the longitudinal length is h and the transverse length is R, judging the prediction mode of the candidate image block. If the mode is the intra-frame mode, no template is generated, or the template is filled according to a default value to be used as a left template. If the image block is in the inter-frame mode, motion information (such as a motion vector and a reference frame index) of the candidate image block is acquired, and a template with the transverse length of R and the longitudinal length of h is generated as a left template based on the motion vector and the reference frame index.
Case three: referring to fig. 2G, the candidate image blocks may include a first inter-mode adjacent image block on the upper side of the current image block and a first inter-mode adjacent image block on the left side of the current image block.
Compared with fig. 2F in the second case, in the third case, the horizontal length W of the upper template of the current image block a1 is taken as the horizontal length of the current image block a1, and the vertical length H of the left template of the current image block a1 is taken as the vertical length of the current image block a1, which are similar to those in the second case and are not repeated.
Case four: referring to fig. 2H, the candidate image blocks may include an inter-mode adjacent image block on the upper side of the current image block, a sub-adjacent image block on the upper side of the current image block (i.e., when the adjacent image block is in the intra-frame mode, the image block corresponding to the adjacent image block is selected to be the sub-adjacent image block to the current image block), an inter-mode adjacent image block on the left side of the current image block, and a sub-adjacent image block on the left side of the current image block.
For example, for the current block a1, there are neighboring blocks in the inter mode on the left side, such as block A3 and block a4, the blocks A3 and a4 in the inter mode may be determined as candidate blocks for the current block a 1. Similarly, if there is an adjacent image block in the inter mode, such as the image block a2, on the upper side, the image block a2 in the inter mode may be determined as a candidate image block for the current image block a 1.
Further, an adjacent image block in intra mode, such as image block a7, exists on the left, and image block a7 has image block A8 in inter mode on the left, i.e., image block A8 is the next adjacent image block to the left of the current image block a1, and thus, image block A8 in inter mode may be determined as a candidate image block for the current image block a 1.
Similarly, an adjacent image block in intra mode, such as image block a5, exists on the upper side, and image block a5 has image block a6 in inter mode on the upper side, i.e., image block a6 is the next adjacent image block on the upper side of the current image block a1, and thus, image block a6 in inter mode may be determined as a candidate image block for the current image block a 1.
After determining that the candidate image block is the image block a2, the image block A3, the image block a4, the image block A6, and the image block A8, a template of the current image block a1 is obtained according to the motion information of the image block a2, the motion information of the image block A3, the motion information of the image block a4, the motion information of the image block A6, and the motion information of the image block A8, where the specific obtaining manner is as in the case one, only obtaining the template according to the motion information of the image block A6 and the motion information of the image block A8 is needed, and is not described herein, and the template is finally shown in fig. 2I.
Case five: referring to fig. 2J, if the first adjacent image block on the upper side of the current image block is in the intra mode and the image block on the upper side of the first adjacent image block is in the inter mode, the candidate image blocks may include a second adjacent image block in the inter mode on the upper side of the current image block. In addition, if the first adjacent image block on the left side of the current image block is in the intra mode and the image block on the left side of the first adjacent image block is in the inter mode, the candidate image blocks may include a second adjacent image block in the inter mode on the left side of the current image block.
For the current block a1, if the first block a2 in the upper side is in intra mode and there is an inter-mode block a4 in the upper side of block a2, i.e., block a4 is the next adjacent block in the upper side of the current block a1, then block a4 in inter mode may be determined as a candidate block for the current block a 1.
Further, if the first tile A3 on the left is in intra mode and there is a tile A5 in inter mode on the left of tile A3, i.e., tile A5 is the next adjacent tile to the left of the current tile A1, then tile A5 in inter mode may be determined to be a candidate tile for the current tile A1.
After determining that the candidate image blocks are the image block a4 and the image block a5, a template of the current image block a1 may be obtained according to the motion information of the image block a4 and the motion information of the image block a5, where the specific obtaining manner is referred to as case two and case three, which is not described herein again, and the template is finally referred to as shown in fig. 2K or fig. 2L.
Case six: referring to fig. 2H, the candidate image blocks may include neighboring image blocks of an inter mode on an upper side of the current image block, neighboring image blocks of an intra mode on an upper side of the current image block, neighboring image blocks of an inter mode on a left side of the current image block, and neighboring image blocks of an intra mode on a left side of the current image block.
For example, for the current block a1, there are neighboring blocks in the inter mode on the left side, such as block A3 and block a4, the blocks A3 and a4 in the inter mode may be determined as candidate blocks for the current block a 1. Similarly, if there is an adjacent image block in the inter mode, such as the image block a2, on the upper side, the image block a2 in the inter mode may be determined as a candidate image block for the current image block a 1.
The neighboring tile of the left side in intra mode, such as tile a7, may determine tile a7 in intra mode as a candidate tile for current tile a 1. The neighboring image block having the intra mode at the upper side, such as the image block a5, may determine the image block a5 of the intra mode as a candidate image block of the current image block a 1.
After determining that the candidate image block is the image block a2, the image block A3, the image block a4, the image block a5, and the image block a7, a template of the current image block a1 is obtained according to the motion information of the image block a2, the motion information of the image block A3, the motion information of the image block a4, the motion information of the image block a5, and the motion information of the image block a7, where the specific obtaining manner is as in the case one, only obtaining the template according to the motion information of the image block a5 and the motion information of the image block a7 is needed, and is not described herein, and the template is finally shown in fig. 2M.
It should be noted that, when the template is obtained according to the motion information of the image block a5, since the image block a5 is in intra mode and has no motion information, the motion information of an adjacent image block (i.e., the image block a6) of the image block a5 may be determined as the motion information of the image block a5, and after the motion information of the image block a5 is obtained, the template may be obtained according to the motion information of the image block a5, as shown in the case one.
Similarly, when the template is acquired based on the motion information of the image block a7, since the image block a7 is in intra mode and has no motion information, it is possible to determine the motion information of an adjacent image block to the image block a7 (i.e., the image block A8) as the motion information of the image block a5 and acquire the template using the motion information of the image block a 5.
Case seven: referring to fig. 2J, if the first adjacent image block on the upper side of the current image block is in the intra mode and the image block on the upper side of the first adjacent image block is in the inter mode, the candidate image blocks may include the adjacent image block in the first intra mode on the upper side of the current image block. In addition, if the first neighboring image block on the left side of the current image block is in intra mode and the image block on the left side of the first neighboring image block is in inter mode, the candidate image blocks may include the neighboring image block on the left side of the current image block in the first intra mode.
For the current block a1, if the first block a2 on the top side is in intra mode, block a4 in inter mode exists on the top side of block a2, and block a2 in intra mode is determined to be a candidate block for the current block a 1. If the first image block A3 on the left is in intra mode, there is an image block A5 in inter mode on the left side of image block A3, and the image block A3 in intra mode is determined to be a candidate image block for the current image block A1.
After determining that the candidate image blocks are the image block a2 and the image block A3, a template of the current image block a1 may be obtained according to the motion information of the image block a2 and the motion information of the image block A3, where the specific obtaining manner is referred to as case two and case three, which is not described herein again, and the template is finally referred to as shown in fig. 2N or fig. 2O.
It is to be noted that, when acquiring the template from the motion information of the image block a2, since the image block a2 is in intra mode and has no motion information, it is possible to determine the motion information of an adjacent image block of the image block a2 (i.e., the image block a4) as the motion information of the image block a2 and acquire the template using the motion information of the image block a 2. When the template is acquired from the motion information of the image block A3, since the image block A3 is in intra mode and has no motion information, it is possible to determine the motion information of an adjacent image block of the image block A3, that is, the image block a5, as the motion information of the image block A3 and acquire the template using the motion information of the image block A3.
Example three:
in the inter-frame coding technique, a relative displacement between a current image block of a current frame video image and a reference image block of a reference frame video image may be represented using a motion vector. For example, video image A is a current frame video image, tile A1 is a current tile, video image B is a reference frame video image for video image A, and tile B1 is a reference tile for tile A1. Due to the strong temporal correlation between video image a and video image B, when image block a1 of video image a needs to be transmitted, a motion search may be performed in video image B to find image block B1 that best matches image block a1, and determine the relative displacement between image block a1 and image block B1, which is also the motion vector of image block a 1.
For example, the motion vector is (-6, 4), which indicates that image block B1 is shifted by 6 pixels to the left in the horizontal direction and by 4 pixels to the top in the vertical direction, compared to image block a 1.
When the encoding end sends the encoded bitstream to the decoding end, the encoded bitstream carries the motion vector (-6, 4) of the image block a1, instead of the image block a 1. The motion vector (-6, 4) for image block a1 may be obtained by the decoder after receiving the encoded bitstream, and the position of image block B1 is determined in video picture B based on the position of image block a1 and the motion vector (-6, 4), i.e.: at the position of the image block a1, 6 pixel points are moved to the left, 4 pixel points are moved upwards, the position of the image block B1 is obtained, the image block B1 is read from the position of the image block B1, and the image block a1 is reconstructed by using the image block B1. Since the similarity between the image block B1 and the image block a1 is high, an image with high similarity can be reconstructed by reconstructing the image block a1 using the image block B1. Since the number of bits occupied by the motion vector is smaller than the number of bits occupied by image block a1, a significant amount of bits is saved by carrying the motion vector in the encoded bitstream instead of carrying image block a 1.
Further, if the video image a includes a large number of image blocks, the motion vector of each image block also occupies relatively many bits, and therefore, in order to further save the number of bits, in the embodiment of the present application, the motion vector of the image block a1 may also be determined by using the motion vectors of adjacent image blocks. For example, in video image A, the image blocks adjacent to image block A1 may include: the image block a2 and the image block A3 may include the motion vector a21 of the image block a2 and the motion vector a31 of the image block A3 in the motion vector list.
When the encoding end sends the encoded bitstream to the decoding end, the encoded bitstream carries the index value of the original motion vector a21 (i.e., the index value in the motion vector list), instead of the motion vector (-6, 4) of image block a1, and even instead of image block a 1. The decoding end receives the encoded bitstream, obtains the index value of the original motion vector a21, and obtains the original motion vector a21 from the motion vector list according to the index value. Since the number of bits occupied by the index value is smaller than the number of bits occupied by the motion vector, bits can be further saved.
In the present embodiment, instead of directly using the original motion vector a21 as the final motion vector of the image block a1, a target motion vector different from the original motion vector a21 is obtained from the original motion vector a21, and the target motion vector is closest to the motion vector of the image block a1, and therefore, the target motion vector can be used as the final motion vector of the image block a 1. Obviously, compared with the mode of directly taking the original motion vector a21 as the final motion vector of the image block a1, the mode of taking the target motion vector as the final motion vector of the image block a1 can solve the problems of low prediction quality, prediction error and the like.
In summary, in this embodiment, the original mode information is an original motion vector of the current image block, the target mode information is a target motion vector of the current image block, the encoding end may obtain the original motion vector corresponding to the current image block, and obtain a target motion vector based on the template according to the original motion vector and the obtained template, where the target motion vector is different from the original motion vector. And coding the bit stream according to the target motion vector to obtain a coded bit stream. The encoding side may transmit the encoded bitstream to the decoding side.
In an example, the encoding end may first obtain an original motion vector of the current image block, and assuming that the current image block is the image block a1, the motion vector list at the encoding end sequentially includes the motion vector a21, the motion vector a31, the motion vector a41, and the motion vector a51, and then one motion vector is selected from the motion vector list, and the selected motion vector is the original motion vector of the image block a 1. Of course, the above manner is only an example, and is not limited to this, for example, the default motion vector may be directly determined as the original motion vector.
Wherein, the selecting, by the encoding end, a motion vector from the motion vector list may include: the encoding end selects a first motion vector from the motion vector list; or, selecting the last motion vector from the motion vector list; or, randomly selecting a first motion vector from the motion vector list; alternatively, a hash algorithm is used to select the first motion vector from the motion vector list. Of course, the above-described modes are only examples, and the modes are not limited thereto as long as a motion vector can be selected from the motion vector list.
The motion vector list is used for recording motion vectors of image blocks adjacent to the current image block. For example, after obtaining the motion vector a21 of the image block a2, the motion vector a21 may be recorded in the motion vector list, after obtaining the motion vector a31 of the image block A3, the motion vector a31 may be recorded in the motion vector list, and so on, and finally, the motion vector list of the image block a1 may be obtained.
In an example, for a process in which the encoding end obtains a target motion vector based on the template according to the original motion vector and the acquired template, reference may be made to subsequent fourth embodiment and fifth embodiment, which are not described herein again. For example, assuming that the motion vector a21 is determined as the original motion vector, a target motion vector may be acquired using the original motion vector a21 as the final motion vector of the image block a 1.
In an example, a process for encoding, by an encoding end, a bitstream according to the target motion vector to obtain an encoded bitstream and sending the encoded bitstream to a decoding end may include: after the target motion vector is obtained, the target motion vector can be used for coding the current image block, the coding mode is not limited, and after the coding is finished, a coded bit stream can be obtained and sent to a decoding end.
Wherein, the encoding end can send the encoded bit stream to the decoding end according to the original motion vector and the target motion vector. Specifically, the encoding performance of the original motion vector and the encoding performance of the target motion vector can be obtained. And when the coding performance of the target motion vector is better than that of the original motion vector, sending the coding bit stream carrying the first indication information to a decoding end. And when the encoding performance of the original motion vector is better than that of the target motion vector, transmitting the encoding bit stream carrying the second indication information to a decoding end.
For the process of acquiring the encoding performance of the original motion vector and the encoding performance of the target motion vector, reference may be made to the subsequent embodiments, which are not described in detail in this embodiment.
In practical application, the first indication information or the second indication information may also be notified in an implicit manner, that is, the first indication information or the second indication information is not carried in the coded bitstream. Specifically, the encoding end and the decoding end may also negotiate a decision policy or define a policy in a standard, and store the policy in the encoding end and the decoding end, where the decision policy may be default first policy information; or defaulting the second policy information; or, the third policy information is defaulted, and the third policy information is the same policy information as that adopted by the adjacent image block of the current image block. When the decision policy is the default first policy information and the coding performance of the target motion vector is better than that of the original motion vector, the first indication information and the second indication information may not be carried in the coded bitstream. When the decision policy is the default second indication information and the encoding performance of the original motion vector is better than that of the target motion vector, the first indication information and the second indication information may not be carried in the encoded bitstream. When the decision strategy is the default third strategy information and the adjacent image block adopts the first strategy information, and the coding performance of the target motion vector is superior to that of the original motion vector, the coded bit stream may not carry the first indication information and the second indication information; when the adjacent image blocks adopt the second policy information and the encoding performance of the original motion vector is better than that of the target motion vector, the encoded bitstream may not carry the first indication information and the second indication information.
In one example, the transmitting, by the encoding end, the encoded bitstream to the decoding end according to the original motion vector and the target motion vector may further include: acquiring an index value of an original motion vector in a motion vector list; and sending the coded bit stream carrying the index value to a decoding end. For example, if the original motion vector is motion vector a21 and motion vector a21 is the first motion vector of the motion vector list, the index value may be 1.
According to the technical scheme, the target motion vector can be obtained according to the original motion vector, the final motion vector of the current image block is determined according to the target motion vector, and the final motion vector of the current image block is not directly determined according to the original motion vector, so that the precision of the motion vector is improved, and the coding performance is improved. And when the target motion vector is obtained according to the original motion vector, the template of the current image block can be obtained according to the motion information of the candidate image block, and the target motion vector is obtained according to the template of the current image block. For example, before the reconstruction stage of decoding, the template of the current image block may be obtained, and the target motion vector may be obtained according to the template of the current image block.
Example four:
on the basis of the third embodiment, for a process in which the encoding end obtains a target motion vector based on a template according to the original motion vector and the acquired template, as shown in fig. 3A, the process may include:
in step 311, the encoding end determines the original motion vector as the central motion vector.
In step 312, the encoding end determines an edge motion vector corresponding to the center motion vector.
Wherein the edge motion vector may be different from the center motion vector.
The determining, by the encoding end, the edge motion vector corresponding to the center motion vector may include: and shifting the central motion vector (x, y) to different directions by S, thereby obtaining edge motion vectors (x-S, y), edge motion vectors (x + S, y), edge motion vectors (x, y + S) and edge motion vectors (x, y-S) in different directions. For example, in the horizontal direction, the center motion vector (x, y) may be shifted to the left by S, resulting in an edge motion vector (x-S, y); in the horizontal direction, the central motion vector (x, y) may be shifted to the right by S, resulting in an edge motion vector (x + S, y); in the vertical direction, the central motion vector (x, y) may be shifted upwards by S, resulting in an edge motion vector (x, y + S); in the vertical direction, the center motion vector (x, y) may be shifted downward by S, resulting in an edge motion vector (x, y-S).
The initial value of S may be empirically configured, and may be 2, 4, 8, 16, etc.
Assuming that the center motion vector is (3, 3) and S is 4, the edge motion vectors are edge motion vector (7, 3), edge motion vector (3, 7), edge motion vector (-1, 3), edge motion vector (3, -1).
Step 313, the encoding end obtains the encoding performance of the central motion vector according to the template of the current image block, and obtains the encoding performance of the edge motion vector according to the template of the current image block.
In the first case, the encoding end obtains the encoding performance of the central motion vector according to the template of the current image block, which may include but is not limited to: and determining the coding performance of the central motion vector according to the parameter information of the template of the current image block and the parameter information of the first target reference block, wherein the first target reference block can be an image block obtained after the reference image block corresponding to the template is subjected to offset based on the central motion vector. Specifically, the encoding end may determine the prediction performance of the central motion vector according to the parameter information of the template and the parameter information of the first target reference block, and determine the encoding performance of the central motion vector according to the prediction performance of the central motion vector; for example, the encoding performance of the center motion vector may be determined according to the prediction performance and the actual number of bits required for encoding.
Wherein, the parameter information may be a brightness value; alternatively, there may be luminance values and chrominance values.
Assuming that the parameter information is a luminance value, in order to determine the encoding performance of the central motion vector, a luminance value of a template of the current image block and a luminance value of the first target reference block may be obtained first. For example, after obtaining the template of the current image block, the luminance value of each pixel point of the template may be obtained, and a reference image block of the template may be obtained, assuming that the central motion vector is (3, 3), the reference image block may be moved by using the central motion vector (3, 3), and an image block X corresponding to the reference image block is obtained (for example, the reference image block is moved rightward by 3 pixel points, the processed image block is recorded as the image block X, and the image block X is the first target reference block, and the luminance value of each pixel point of the image block X may be obtained.
Based on the brightness value of each pixel point of the template and the brightness value of each pixel point of the image block X, the prediction performance of the central motion vector can be determined by adopting the following formula:
Figure BDA0001712805380000241
SAD is the sum of the available absolute differences for representing the prediction performance of the central motion vector. TMiThe brightness value, TMP, of the ith pixel point of the templateiAnd the brightness value of the ith pixel point of the image block X is represented, and M represents the total number of the pixel points.
Assuming that the parameter information is a luminance value and a chrominance value, the following formula is used
Figure BDA0001712805380000242
Determining the brightness value prediction performance SAD of the central motion vector, and determining the chroma value prediction performance of the central motion vector by adopting the following formula:
Figure BDA0001712805380000243
the average of the luminance value prediction performance SAD and the chrominance value prediction performance CSAD is the prediction performance of the central motion vector. Where CSAD is the sum of available absolute differences for representing chroma value prediction performance, CTM, of the central motion vectoriExpressing the chroma value, CTMP, of the ith pixel point of the templateiExpressing the colorimetric value, M, of the ith pixel point of the image block XcIndicating the total number of pixels.
Further, after the prediction performance of the central motion vector is obtained, the coding performance of the central motion vector may be determined according to the prediction performance and the actual number of bits required for coding. For example, RDO (Rate Distortion Optimized) may be used to determine the coding performance of the center motion vector, and RDO generally determines the coding performance of the center motion vector by using the following formula: j ═ D + λ R. Wherein J represents the coding performance, D represents the prediction performance, λ is a lagrange multiplier, which is a value configured according to experience, and R is the actual number of bits required for image block coding, i.e., the sum of bits of the coded bitstream carrying information.
In case two, the encoding end obtains the encoding performance of the edge motion vector according to the template of the current image block, which may include but is not limited to: and determining the encoding performance of the edge motion vector according to the parameter information of the template of the current image block and the parameter information of the second target reference block, wherein the second target reference block can be an image block obtained after the reference image block corresponding to the template is subjected to offset based on the edge motion vector. Specifically, the encoding end may determine the prediction performance of the edge motion vector according to the parameter information of the template and the parameter information of the second target reference block, and determine the encoding performance of the central motion vector according to the prediction performance of the edge motion vector; for example, the encoding performance of the edge motion vector may be determined according to the prediction performance and the actual number of bits required for encoding.
Wherein, the parameter information may be a brightness value; alternatively, there may be luminance values and chrominance values.
Case two is similar to case one, with the difference that: in case one, the reference image block of the template is moved by using the edge motion vector to obtain a second target reference block, and the coding performance of the edge motion vector is obtained by using the second target reference block, while in case two, the reference image block of the template is moved by using the center motion vector to obtain a first target reference block, and the coding performance of the center motion vector is obtained by using the first target reference block.
In step 314, the encoding end determines the target motion vector from the center motion vector and the edge motion vector according to the encoding performance of the center motion vector and the encoding performance of the edge motion vector.
Specifically, the encoding end may select a motion vector with the optimal encoding performance from the center motion vector and the edge motion vector; when the motion vector with the optimal coding performance is not the original motion vector, the motion vector with the optimal coding performance can be determined as a target motion vector; when the motion vector with the optimal coding performance is the original motion vector, the coding end can select a motion vector with the suboptimal coding performance from the central motion vector and the edge motion vector, and determine the motion vector with the suboptimal coding performance as the target motion vector.
For example, if the motion vector with the best encoding performance is the edge motion vector (7, 3), the encoding end may determine the edge motion vector (7, 3) as the target motion vector. If the motion vector with the best coding performance is the central motion vector (3, 3), that is, the original motion vector, the coding end may also determine the motion vector with the suboptimal coding performance (such as the edge motion vector (7, 3), etc.) as the target motion vector.
Example five:
on the basis of the third embodiment, for a process in which the encoding end obtains a target motion vector based on a template according to the original motion vector and the acquired template, as shown in fig. 3B, the process may include:
in step 321, the encoding end determines the original motion vector as the central motion vector.
In step 322, the encoding end determines an edge motion vector corresponding to the center motion vector.
Wherein the edge motion vector may be different from the center motion vector.
Step 323, the encoding end obtains the encoding performance of the central motion vector according to the template of the current image block, and obtains the encoding performance of the edge motion vector according to the template of the current image block.
In step 324, the encoding end determines whether an iteration end condition of the target motion vector is satisfied.
If so, step 326 may be performed; if not, step 325 may be performed.
Wherein, the iteration end condition may include but is not limited to: the number of iterations reaches a threshold number, or the execution time reaches a time threshold, or the parameter S has been modified to a preset value, such as 1.
Of course, the above is only a few examples of the iteration end condition, and the iteration end condition is not limited.
Step 325, the encoding end selects the motion vector with the best encoding performance from the central motion vector and the edge motion vector, determines the motion vector with the best encoding performance as the central motion vector, and returns to step 322.
For example, if the motion vector with the best coding performance is the edge motion vector (7, 3), the edge motion vector (7, 3) may be determined as the center motion vector, and step 322 may be performed again, and so on.
When step 322 is performed for the first time, the value of the parameter S may be an initial value, such as 16. When step 322 is executed again, the value of the parameter S is first adjusted, for example, to be the last parameter S minus 2, or to be half of the last parameter S, and the like, which is not limited to this, as long as it is smaller than the last parameter S, and then the adjustment to be half of the last parameter S is taken as an example. Therefore, when step 322 is performed for the second time, the value of the parameter S is 8; when step 322 is executed for the third time, the value of the parameter S is 4; and so on.
After the value of the parameter S is adjusted, it is first determined whether the adjusted parameter S is less than or equal to a preset value, such as 1. If not, step 322 may be executed based on the adjusted parameter S, which is not described in detail herein. If so, the value of the parameter S may be set to 1, and step 322 is executed based on the parameter S (i.e., the value of 1), and when the step 324 is executed, the iteration end condition is satisfied.
In step 326, the encoding end determines the target motion vector from the center motion vector and the edge motion vector according to the encoding performance of the center motion vector and the encoding performance of the edge motion vector.
Example six:
in the third embodiment, the encoding end may obtain the encoding performance of the original motion vector and the encoding performance of the target motion vector. The obtaining, by the encoding end, the encoding performance of the original motion vector may include: and determining the encoding performance of the original motion vector according to the parameter information of the template of the current image block and the parameter information of a third target reference block, wherein the third target reference block is an image block obtained after the reference image block corresponding to the template is subjected to offset based on the original motion vector. Specifically, the prediction performance of the original motion vector may be determined according to the parameter information of the template and the parameter information of the third target reference block, and the encoding performance of the original motion vector may be determined according to the prediction performance; for example, the encoding performance of the original motion vector is determined according to the prediction performance and the actual number of bits required for encoding. Wherein, the parameter information may be a brightness value; or luminance values and chrominance values.
The encoding performance of the encoding end for obtaining the target motion vector may include, but is not limited to: and determining the coding performance of the target motion vector according to the parameter information of the template of the current image block and the parameter information of a fourth target reference block, wherein the fourth target reference block is an image block obtained after the reference image block corresponding to the template is subjected to offset based on the target motion vector. Specifically, the prediction performance of the target motion vector may be determined according to the parameter information of the template and the parameter information of the fourth target reference block, and the encoding performance of the target motion vector may be determined according to the prediction performance; for example, the encoding performance of the target motion vector is determined based on the prediction performance and the actual number of bits required for encoding. Wherein, the parameter information may be a brightness value; or luminance values and chrominance values.
The above process is similar to the third embodiment, except that: when the third target reference block or the fourth target reference block is obtained, the original motion vector or the target motion vector is used to move the reference image block of the template, instead of the reference image block of the central motion vector moving template, and the details are not repeated here.
Example seven:
in this embodiment, the original mode information is an original motion vector and an original reference frame of the current image block, and the target mode information is a target motion vector and a target reference frame of the current image block. Based on this, the encoding end may obtain an original motion vector and an original reference frame corresponding to the current image block, and obtain a target motion vector (which may be different from the original motion vector) and a target reference frame based on the template according to the original motion vector, the original reference frame, and the obtained template. Then, the bitstream can be encoded according to the target motion vector and the target reference frame to obtain an encoded bitstream, and the encoded bitstream is sent to a decoding end.
In an example, the encoding end may first obtain an original motion vector of the current image block, and assuming that the current image block is the image block a1, the motion vector list at the encoding end sequentially includes the motion vector a21, the motion vector a31, the motion vector a41, and the motion vector a51, and then one motion vector is selected from the motion vector list, and the selected motion vector is the original motion vector of the image block a 1. Of course, the above manner is only an example, and is not limited to this, for example, the default motion vector may be directly determined as the original motion vector.
In an example, in a process of acquiring an original reference frame of a current image block, for the current image block, there may be one or more reference frames (video frames with strong temporal correlation), one of the reference frames may be used as the original reference frame, and the remaining reference frames are candidate reference frames.
In an example, the process of obtaining, by the encoding end, the target motion vector and the target reference frame based on the template according to the original motion vector, the original reference frame, and the obtained template may refer to the subsequent embodiments, and details are not described herein. For example, if the motion vector a21 is determined as an original motion vector, the reference frame 1 is determined as an original reference frame, and the reference frames 2 and 3 are determined as candidate reference frames, then the original motion vector a21, the reference frame 1, the reference frame 2, and the reference frame 3 are used to obtain a target motion vector and a target reference frame, the target motion vector is used as a final motion vector of the current image block, the target reference frame may be one of the reference frame 1, the reference frame 2, and the reference frame 3, and the target reference frame is used as a final reference frame of the current image block.
In an example, the process of encoding, by the encoding end, the bitstream according to the target motion vector and the target reference frame to obtain an encoded bitstream, and sending the encoded bitstream to the decoding end may include, but is not limited to: after obtaining the target motion vector and the target reference frame, the encoding end may encode the current image block by using the target motion vector and the target reference frame, the encoding method is not limited, and after the encoding is completed, the encoding end may obtain an encoded bit stream and send the encoded bit stream to the decoding end.
Wherein, the encoding end can send the encoded bit stream to the decoding end according to the original motion vector and the target motion vector. Specifically, the encoding performance of the original motion vector and the encoding performance of the target motion vector can be obtained. And when the coding performance of the target motion vector is better than that of the original motion vector, sending the coding bit stream carrying the first indication information to a decoding end. And when the encoding performance of the original motion vector is better than that of the target motion vector, transmitting the encoding bit stream carrying the second indication information to a decoding end.
For the process of acquiring the encoding performance of the original motion vector and the encoding performance of the target motion vector, reference may be made to the sixth embodiment described above, and details of this process are not described in this embodiment again.
In practical application, the first indication information or the second indication information may also be notified in an implicit manner, that is, the first indication information or the second indication information is not carried in the coded bit stream, specifically, the coding end and the decoding end may also negotiate a decision policy or define a policy in a standard, and store the policy in the coding end and the decoding end, for example, the decision policy may be the default first policy information; or defaulting the second policy information; or, the third policy information is defaulted, and the third policy information is the same policy information as that adopted by the adjacent image block of the current image block. Then, it may be determined, based on the decision policy, in which case the first indication information or the second indication information may not be carried in the coded bitstream, and the detailed processing is referred to as embodiment three and is not described herein again.
In one example, the transmitting, by the encoding end, the encoded bitstream to the decoding end according to the original motion vector and the target motion vector may further include: acquiring an index value of an original motion vector in a motion vector list; and sending the coded bit stream carrying the index value to a decoding end. For example, if the original motion vector is motion vector a21 and motion vector a21 is the first motion vector of the motion vector list, the index value may be 1.
According to the technical scheme, the target motion vector and the target reference frame can be obtained according to the original motion vector and the original reference frame, the final motion vector of the current image block is determined according to the target motion vector, the final reference frame of the current image block is determined according to the target reference frame instead of directly determining the final motion vector of the current image block according to the original motion vector, and the final reference frame of the current image block is determined by using the original reference frame, so that the precision of the motion vector is improved, and the coding performance is improved. Moreover, the template of the current image block can be obtained according to the motion information of the candidate image block, the template of the current image block can be quickly obtained, the coding efficiency can be improved, and the coding delay can be reduced.
Example eight:
on the basis of the seventh embodiment, the encoding end obtains a target motion vector and a target reference frame based on the template according to the original motion vector, the original reference frame and the template, and the implementation flow can be shown in fig. 3C.
In step 331, the encoding end obtains a candidate motion vector corresponding to the original reference frame according to the original motion vector based on the template of the current image block, where the candidate motion vector may be different from the original motion vector.
In an example, the encoding end obtains a candidate motion vector corresponding to the original reference frame according to the original motion vector based on the template of the current image block, which may include but is not limited to: the encoding end determines the original motion vector as a central motion vector and determines an edge motion vector corresponding to the central motion vector, wherein the edge motion vector is different from the central motion vector; the encoding end obtains the encoding performance of the central motion vector according to the template of the current image block and obtains the encoding performance of the edge motion vector according to the template of the current image block; then, the encoding end can determine a candidate motion vector corresponding to the original reference frame from the center motion vector and the edge motion vector according to the encoding performance of the center motion vector and the encoding performance of the edge motion vector.
For the process of obtaining the candidate motion vector of the original reference frame by the encoding end according to the original motion vector, reference may be made to embodiment four or embodiment five, but only the target motion vector in embodiment four or embodiment five is replaced by the candidate motion vector of the original reference frame, and details are not repeated here.
In step 332, the encoding end obtains an initial motion vector corresponding to each candidate reference frame (e.g., each candidate reference frame in one or more candidate reference frames) according to the original motion vector.
In an example, the obtaining, by the encoding end, an initial motion vector corresponding to each candidate reference frame according to the original motion vector may include: the first candidate reference frame is any one of candidate reference frames, and for the first candidate reference frame, an initial motion vector of the first candidate reference frame may be obtained according to a distance between a frame where the current image block is located and the original reference frame (for example, the distance may be a number of frames between the frame where the current image block is located and the original reference frame), a distance between the frame where the current image block is located and the first candidate reference frame, and the original motion vector.
For example, assuming that the original motion vector is motion vector 1, the original reference frame is reference frame 1, the candidate reference frames are reference frame 2 and reference frame 3, the distance between the frame where the current image block is located (hereinafter referred to as the current frame) and reference frame 1 is d1, the distance between the current frame and reference frame 2 is d2, and the distance between the current frame and reference frame 3 is d3, then: the initial motion vector corresponding to the reference frame 2 is the motion vector 1 x (d2/d1) and is subsequently denoted as the motion vector 2, and the initial motion vector corresponding to the reference frame 3 is the motion vector 1 x (d3/d1) and is subsequently denoted as the motion vector 3.
And 333, the encoding end acquires a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector.
In an example, the encoding end obtains a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector, which may include but is not limited to: the first candidate reference frame may be any one of candidate reference frames, and for the first candidate reference frame, the encoding end may determine an initial motion vector of the first candidate reference frame as a central motion vector, and determine an edge motion vector corresponding to the central motion vector, where the edge motion vector is different from the central motion vector; the encoding end can obtain the encoding performance of the central motion vector according to the template of the current image block and obtain the encoding performance of the edge motion vector according to the template of the current image block; then, the encoding end may determine a candidate motion vector corresponding to the first candidate reference frame from the center motion vector and the edge motion vector according to the encoding performance of the center motion vector and the encoding performance of the edge motion vector.
For each candidate reference frame, the candidate motion vector of the candidate reference frame may be obtained according to the initial motion vector of the candidate reference frame, that is, each candidate reference frame corresponds to one candidate motion vector.
For a process of obtaining the candidate motion vector of the candidate reference frame by the encoding end according to the initial motion vector of the candidate reference frame, reference may be made to embodiment four or embodiment five, where only the target motion vector in embodiment four or embodiment five is replaced by the candidate motion vector of the candidate reference frame, and the original motion vector in embodiment four or embodiment five is replaced by the initial motion vector of the candidate reference frame, which is not described herein again.
In step 334, the encoding end selects a candidate motion vector with the best encoding performance from the candidate motion vectors corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames.
For example, after obtaining the candidate motion vector corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames, the encoding end may obtain the encoding performance of each candidate motion vector, and the specific obtaining manner refers to the fourth embodiment, and it is sufficient to replace the center motion vector of the fourth embodiment with the candidate motion vector, which is not described herein again. After the encoding end obtains the encoding performance of each candidate motion vector, the encoding end can select the candidate motion vector with the optimal encoding performance, and the selection process is not repeated.
Step 335, the encoding end determines the candidate motion vector with the best encoding performance as the target motion vector, and determines the reference frame corresponding to the candidate motion vector with the best encoding performance as the target reference frame.
For example, when the candidate motion vector with the best coding performance corresponds to the original reference frame, the coding end may determine the original reference frame as the target reference frame, and when the candidate motion vector with the best coding performance corresponds to the candidate reference frame, the coding end may determine the candidate reference frame as the target reference frame.
In this embodiment, a target reference frame needs to be selected from the original reference frame and all candidate reference frames, and the target reference frame is the final reference frame of the current image block. A target motion vector (i.e., a candidate motion vector with the best coding performance) needs to be selected from candidate motion vectors corresponding to an original reference frame and candidate motion vectors corresponding to all candidate reference frames, and the target motion vector is a final motion vector of a current image block.
Example nine:
in this embodiment, the original mode information may be an original predicted image block, the target mode information may be a target predicted image block, and the encoding end may obtain the target predicted image block according to the original predicted image block.
Specifically, in the inter-frame coding technique, in order to transmit a current image block of a current frame video image, a reference frame video image corresponding to the current image block may be queried first, that is, the reference frame video image may be queried through a reference frame index, then a prediction image block matching the current image block is queried from the reference frame video image, and when the current image block needs to be coded, the current image block is not directly coded, but a difference between the current image block and the prediction image block is coded, so that the bit transmission amount is reduced.
In one example, the difference between the current image block and the predicted image block may be directly encoded, but since there may be a luminance difference between the current image block and the predicted image block, when the difference between the current image block and the predicted image block is directly encoded, there are problems of poor encoding effect, low encoding performance, and the like.
Different from the above manner, in this embodiment, a prediction image block in a reference frame video image is referred to as an original prediction image block, a target prediction image block is obtained according to the original prediction image block, and a difference between a current image block and the target prediction image block is encoded, instead of directly encoding the difference between the current image block and the original prediction image block. The target prediction image block is an image block obtained by adopting a local illumination compensation algorithm, so that the brightness difference between the prediction image block and the current image block can be eliminated, and the problems of poor coding effect, low coding performance and the like can be solved when the difference between the current image block and the target prediction image block is coded.
In this embodiment, the encoding end may obtain an original predicted image block corresponding to the current image block, and obtain a target predicted image block based on the template according to the original predicted image block and the obtained template. And coding the bit stream according to the target prediction image block to obtain a coded bit stream, and sending the coded bit stream to a decoding end.
In an example, the process of obtaining, by the encoding side, an original predicted image block corresponding to the current image block may include, but is not limited to, the following ways: the encoding end firstly obtains a reference frame index corresponding to the current image block, and inquires a reference frame video image corresponding to the current image block through the reference frame index, and then the encoding end can inquire an original prediction image block matched with the current image block from the reference frame video image.
In an example, the process of obtaining the target prediction image block based on the template by the encoding end according to the original prediction image block and the obtained template may refer to the subsequent embodiments, and is not described herein again.
In an example, the process of encoding, by an encoding end, a bitstream according to the target prediction image block to obtain an encoded bitstream, and sending the encoded bitstream to a decoding end may include: after the target prediction image block is obtained, the difference between the current image block and the target prediction image block can be coded, the coding mode is not limited, after the coding is finished, a coded bit stream is obtained, and the coded bit stream is sent to a decoding end.
Further, when the encoding end adopts the target prediction image block to encode the bit stream (namely, the difference between the current image block and the target prediction image block is encoded), the encoding end sends the encoded bit stream carrying the first indication information to the decoding end; when the encoding end adopts the original prediction image block to encode the bit stream (namely, the difference between the current image block and the original prediction image block is encoded), the encoding end sends the encoded bit stream carrying the second indication information to the decoding end.
In practical application, the first indication information or the second indication information may also be notified in an implicit manner, that is, the first indication information or the second indication information is not carried in the coded bit stream, specifically, the coding end and the decoding end may also negotiate a decision policy or define a policy in a standard, and store the policy in the coding end and the decoding end, for example, the decision policy may be the default first policy information; or defaulting the second policy information; or, the third policy information is defaulted, and the third policy information is the same policy information as that adopted by the adjacent image block of the current image block.
For example, if the decision policy is default first policy information and the encoding end encodes the bitstream by using the target prediction image block, the encoded bitstream does not carry the first indication information or the second indication information;
if the decision strategy is the default second strategy information and the encoding end adopts the original prediction image block to encode the bit stream, the encoding bit stream does not carry the first indication information or the second indication information;
if the decision strategy is the default third strategy information, the adjacent image block of the current image block adopts the first strategy information, and the coding end adopts the target prediction image block to code the bit stream, the first indication information or the second indication information is not carried in the coded bit stream; and if the adjacent image block of the current image block adopts the second strategy information and the encoding end adopts the original prediction image block to encode the bit stream, the first indication information or the second indication information is not carried in the encoded bit stream.
Example ten:
on the basis of the ninth embodiment, the encoding end obtains a process of the target prediction image block based on the template according to the original prediction image block and the acquired template, and an implementation flow thereof can be shown in fig. 3D.
Step 341, the encoding end obtains the motion vector of the current image block and the reference frame index corresponding to the motion vector. For the current image block, the motion vector of the current image block may be the original motion vector in the above embodiment, may be the target motion vector in the above embodiment, or may be the final motion vector in the above embodiment, which is not limited to this, as long as the motion vector of the current image block can be obtained. In addition, the reference frame index corresponding to the motion vector is the index of the reference frame of the current image block.
In step 342, the encoding end obtains a reference block corresponding to the template according to the motion vector of the current image block and the reference frame index. For example, a reference frame video image is searched through the reference frame index, a reference image block matched with the current image block is searched from the reference frame video image, then the reference image block is moved by using the motion vector of the current image block, the obtained reference block is the reference block corresponding to the template, and the reference block is subsequently marked as TMP.
In step 343, the encoding end obtains the transform coefficient of the original prediction image block according to the template of the current image block and the reference block corresponding to the template. Specifically, the encoding end may obtain the transform coefficient of the original prediction image block according to the parameter information of each pixel of the template of the current image block and the parameter information of each pixel of the reference block corresponding to the template. The parameter information may include, but is not limited to, a brightness value of each pixel.
See, for example, the following equation:
Figure BDA0001712805380000341
TMirepresenting the luminance value, TMP, of the ith pixel in the template of the current image blockiThe luminance value of the ith pixel in the reference block corresponding to the template is represented, M represents the total number of pixels of the template, a and b are transformation coefficients, and the value of SSE is changed by adjusting the values of a and b. When the value of the SSE is minimum, the values of a and b are target values, namely the values of the transformation coefficients a and b are obtained. Wherein, the SSE value can be minimized by adopting a least square method, and then transformation coefficients a and b are obtainedAnd b is selected. Of course, other algorithms may be used, without limitation.
And 344, the encoding end performs illumination compensation on the original prediction image block according to the transformation coefficient to obtain a target prediction image block. For example, the target predicted image block may be aX + b, with X representing the original predicted image block.
Example eleven:
referring to fig. 4, a flowchart of a decoding method is shown, and the method is applied to a decoding end, and may include the following steps:
step 401, the decoding end obtains motion information of a candidate image block of the current image block.
The candidate image blocks of the current image block may include, but are not limited to: a spatial domain candidate image block of the current image block; or, a time domain candidate image block of the current image block; the candidate image block is not limited.
Step 402, the decoding end obtains the template of the current image block according to the motion information of the candidate image block.
The method for acquiring the template of the current image block by the decoding end according to the motion information of the candidate image block is the same as that of the encoding end, and reference may be specifically made to the related contents of embodiment two, which is not described herein again.
In step 403, the decoding end decodes the encoded bitstream according to the template of the current image block.
In one example, the decoding end decodes the encoded bitstream according to the template of the current image block, which may include but is not limited to: acquiring original mode information of a current image block; acquiring target mode information of the current image block according to the original mode information based on the template of the current image block; the encoded bitstream is decoded according to the target mode information. The method comprises the steps that original mode information is an original motion vector, and target mode information is a target motion vector; or the original mode information is an original motion vector and an original reference frame, and the target mode information is a target motion vector and a target reference frame; or the original mode information is an original prediction image block, and the target mode information is a target prediction image block. Of course, the above is merely an example, and no limitation is made thereto.
In an example, before obtaining the target mode information of the current image block according to the original mode information based on the template of the current image block, the decoding end may further obtain a coded bit stream, where the coded bit stream may be sent by the encoding end, or obtained by the decoding end by encoding the bit stream.
For example, the decoding end may receive the encoded bitstream from the encoding end, and if the encoded bitstream carries first indication information, the first indication information is used to indicate a template based on the current image block, and obtain target mode information of the current image block according to the original mode information; acquiring motion information of candidate image blocks of the current image block according to the first indication information; and acquiring the template of the current image block according to the motion information of the candidate image block.
For another example, the decoding end may receive the encoded bitstream from the encoding end, and if the encoded bitstream carries second indication information, the second indication information is used to indicate that the encoded bitstream is decoded based on the original mode information corresponding to the current image block; then, according to the second indication information, original mode information corresponding to the current image block is obtained, and the encoded bit stream is decoded based on the original mode information.
In practical application, the encoding end may also notify a policy corresponding to the first indication information or the second indication information in an implicit manner, that is, the encoded bitstream does not carry the first indication information or the second indication information. Based on the above, the decoding end can also obtain the motion information of the candidate image block of the current image block according to the locally preset first strategy information; acquiring a template of a current image block according to the motion information of the candidate image block; or, the original mode information may be acquired according to second policy information preset locally, and the encoded bitstream is decoded based on the original mode information; or, the third policy information may also be acquired, and the encoded bitstream is decoded according to the locally preset third policy information; the first strategy information is used for indicating a template based on the current image block, and acquiring target mode information of the current image block according to the original mode information; the second strategy information is used for indicating that the coded bit stream is decoded based on the original mode information corresponding to the current image block; the third strategy information is the strategy information which is the same as the candidate image block of the current image block.
Specifically, the encoding end and the decoding end may also negotiate a decision policy or define a policy in a standard, and store the policy in the encoding end and the decoding end, where the decision policy may be default first policy information; or default second policy information; or default third policy information. On the basis, if the coded bit stream does not carry the first indication information and the second indication information, when the decision strategy is the default first strategy information, the motion information of the candidate image block of the current image block is obtained according to the locally preset first strategy information; acquiring a template of a current image block according to the motion information of the candidate image block; when the decision strategy is the default second strategy information, acquiring original mode information according to the second strategy information preset locally, and decoding the coded bit stream based on the original mode information; when the decision strategy is the default third strategy information, if the adjacent image block adopts the first strategy information, the motion information of the candidate image block of the current image block is obtained; acquiring a template of a current image block according to the motion information of the candidate image block; and if the adjacent image blocks adopt the second strategy information, acquiring original mode information, and decoding the coded bit stream based on the original mode information.
According to the technical scheme, in the embodiment of the application, the template of the current image block can be obtained according to the motion information of the candidate image block of the current image block, and the coded bit stream is decoded according to the template of the current image block. The method can quickly obtain the template of the current image block, solves the problems of low decoding efficiency, large decoding time delay and the like, can improve the decoding efficiency, reduce the decoding time delay and improve the decoding performance.
Example twelve:
in this embodiment, unlike the manner of generating the template by using the reconstruction information and the prediction information, the template of the current image block may be obtained according to the motion information (such as a motion vector and a reference frame index) of the candidate image block of the current image block. Specifically, the decoding end may determine a reference frame image corresponding to the candidate image block according to the reference frame index; and acquiring a reference image block corresponding to the candidate image block from the reference frame image according to the motion vector, and acquiring a template of the current image block according to the reference image block.
The candidate image blocks may include M first candidate image blocks and N second candidate image blocks, where M is a natural number greater than or equal to 1 and N is a natural number greater than or equal to 0, or M is a natural number greater than or equal to 0 and N is a natural number greater than or equal to 1. The first candidate image block is a candidate image block on the upper side of the current image block, and the second candidate image block is a candidate image block on the left side of the current image block. The first candidate image block comprises an adjacent image block and/or a secondary adjacent image block on the upper side of the current image block; the prediction mode of the adjacent image block is an inter mode or an intra mode; the prediction mode of the secondary adjacent image block is an inter mode. The second candidate image block comprises an adjacent image block on the left side of the current image block and/or a secondary adjacent image block; the prediction mode of the adjacent image block is an inter mode or an intra mode; the prediction mode of the secondary adjacent image block is an inter mode.
Obtaining the template of the current image block according to the motion information of the candidate image block may include, but is not limited to: determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks; and determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks. Then, determining the first template as the template of the current image block; or, determining the second template as the template of the current image block; or determining the first template and the second template as the template of the current image block after splicing.
In an example, when M is greater than 1, the first template may include M sub-templates or P sub-templates, and the M sub-templates or the P sub-templates are spliced to form the first template, where P is the number of first candidate image blocks in the inter mode; when M is equal to 1, the first template may comprise a first sub-template determined according to the motion vector prediction mode and the motion information of the first candidate image block on the upper side of the current image block. When N is greater than 1, the second template may include N sub-templates or R sub-templates, and the N sub-templates or R sub-templates are spliced to form the second template, where R is the number of second candidate image blocks in the inter mode; when N is equal to 1, the second template may comprise a second sub-template determined according to the motion vector prediction mode and the motion information of the first candidate image block on the left side of the current image block.
In an example, obtaining the template of the current image block according to the motion information of the candidate image block may further include, but is not limited to: and when the current image block corresponds to a plurality of pieces of motion information, obtaining a template corresponding to the motion information according to each piece of motion information. And acquiring a weight parameter corresponding to each piece of motion information, and acquiring a template of the current image block according to the weight parameter corresponding to each piece of motion information and the template corresponding to the motion information.
The processing procedure in the twelfth embodiment is similar to that in the second embodiment, and thus, the description thereof is not repeated.
Example thirteen:
the original mode information is an original motion vector of a current image block, the target mode information is a target motion vector of the current image block, the decoding end can acquire the original motion vector corresponding to the current image block, and a target motion vector based on the template is obtained according to the original motion vector and the acquired template, wherein the target motion vector is different from the original motion vector. And the decoding end decodes the coded bit stream according to the target motion vector.
For example, the decoding end determines the target motion vector as the final motion vector of the current image block, and decodes the encoded bitstream using the final motion vector of the current image block, without limitation to the decoding process.
The processing procedure for obtaining the target motion vector based on the template by the decoding end according to the original motion vector and the obtained template may refer to the subsequent embodiments, and will not be described herein repeatedly.
In an example, for a process of acquiring an original motion vector of a current image block by a decoding end, the decoding end may further receive a coded bit stream from an encoding end, where the coded bit stream carries an index value of the original motion vector in a motion vector list; the decoding end can select the motion vector corresponding to the index value from the motion vector list; and determining the selected motion vector as the original motion vector corresponding to the current image block. If the index value is 1, the first motion vector of the motion vector list is obtained, and the motion vector is the original motion vector of the current image block. The motion vector list is used for recording motion vectors of image blocks adjacent to the current image block, and the motion vector list maintained by the decoding end is the same as the motion vector list maintained by the encoding end. Of course, the above-mentioned manners are only examples, and for example, the motion vector of the candidate image block of the current image block may be determined as the original motion vector corresponding to the current image block, or the default motion vector may be determined as the original motion vector.
In one example, the decoding end may further parse the first indication information or the second indication information from the encoded bitstream. When the coded bit stream carries the first indication information, a target motion vector based on the template can be obtained according to the original motion vector and the obtained template, and the coded bit stream is decoded by utilizing the target motion vector; when the encoded bitstream carries the second indication information, the encoded bitstream may be decoded from the original motion vector.
According to the technical scheme, the target motion vector can be obtained according to the original motion vector, the final motion vector of the current image block is determined according to the target motion vector, and the final motion vector of the current image block is not directly determined according to the original motion vector, so that the precision of the motion vector is improved, and the decoding performance is improved. Moreover, when the target motion vector is obtained according to the original motion vector, the template of the current image block can be obtained according to the motion information of the candidate image block, and the target motion vector is obtained according to the original motion vector based on the template of the current image block; the method can quickly obtain the template of the current image block, and then obtain the target motion vector according to the template, thereby improving the decoding efficiency and reducing the decoding time delay. For example, before the reconstruction stage of decoding, a template of the current image block may be obtained, and the target motion vector may be obtained according to the template.
Example fourteen:
on the basis of the thirteenth embodiment, a process of obtaining, by a decoding end, a target motion vector based on a template according to an original motion vector and an obtained template is shown in fig. 5, and may include:
in step 501, the decoding end determines the original motion vector as the central motion vector.
In step 502, the decoding end determines an edge motion vector corresponding to the center motion vector.
Wherein the edge motion vector may be different from the center motion vector.
Wherein determining the edge motion vector corresponding to the center motion vector may include: and shifting the central motion vector (x, y) to different directions by S to obtain edge motion vectors (x-S, y), edge motion vectors (x + S, y), edge motion vectors (x, y + S) and edge motion vectors (x, y-S) in different directions.
Step 503, the decoding end obtains the coding performance of the central motion vector according to the template of the current image block, and obtains the coding performance of the edge motion vector according to the template of the current image block.
The decoding end obtains the coding performance of the central motion vector according to the template of the current image block, which may include but is not limited to: the decoding end may determine the encoding performance of the central motion vector according to the parameter information of the template of the current image block and the parameter information of the first target reference block, where the first target reference block is an image block obtained after the reference image block corresponding to the template is shifted based on the central motion vector.
The decoding end obtains the encoding performance of the edge motion vector according to the template of the current image block, which may include but is not limited to: the decoding end may determine the encoding performance of the edge motion vector according to the parameter information of the template of the current image block and the parameter information of the second target reference block, where the second target reference block is an image block obtained after the reference image block corresponding to the template is shifted based on the edge motion vector.
Step 504, the decoding end determines the target motion vector from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
Specifically, the decoding end may select a motion vector with the best coding performance from the center motion vector and the edge motion vector; when the motion vector with the optimal coding performance is not the original motion vector, the motion vector with the optimal coding performance can be determined as a target motion vector; when the motion vector with the optimal coding performance is the original motion vector, the decoding end can select a motion vector with the suboptimal coding performance from the central motion vector and the edge motion vector, and determine the motion vector with the suboptimal coding performance as the target motion vector.
The detailed processing flow of step 501 to step 504 can be seen from the fourth embodiment, except that the execution main body is changed from the encoding end to the decoding end, and other processing flows are the same, and are not repeated herein.
For the decoding end, the target motion vector based on the template is obtained according to the original motion vector and the obtained template, embodiment five may also be adopted, and only the execution main body is changed from the encoding end to the decoding end, which is not described herein again.
Example fifteen:
in this embodiment, the original mode information is an original motion vector and an original reference frame of the current image block, and the target mode information is a target motion vector and a target reference frame of the current image block. Based on this, the decoding end may obtain an original motion vector and an original reference frame corresponding to the current image block, based on the template of the current image block, the decoding end may obtain a target motion vector (the target motion vector may be different from the original motion vector) and a target reference frame based on the template according to the original motion vector and the original reference frame corresponding to the current image block, and decode the encoded bitstream according to the target motion vector and the target reference frame.
The decoding end obtains a target motion vector and a target reference frame based on the template according to the original motion vector corresponding to the current image block and the original reference frame based on the template of the current image block, which may include but is not limited to: based on the template of the current image block, obtaining a candidate motion vector corresponding to the original reference frame according to the original motion vector; acquiring initial motion vectors corresponding to the candidate reference frames according to the initial motion vectors; obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector; then, a candidate motion vector with the optimal coding performance can be selected from the candidate motion vectors corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames; and determining the candidate motion vector with the optimal coding performance as a target motion vector, and determining a reference frame corresponding to the candidate motion vector with the optimal coding performance as a target reference frame.
The obtaining of the initial motion vector corresponding to each candidate reference frame according to the initial motion vector may include, but is not limited to: the first candidate reference frame is any one of the candidate reference frames, and aiming at the first candidate reference frame, the initial motion vector of the first candidate reference frame is obtained according to the distance between the frame of the current image block and the original reference frame, the distance between the frame of the current image block and the first candidate reference frame and the original motion vector.
In an example, based on the template of the current image block, obtaining the candidate motion vector corresponding to the original reference frame according to the original motion vector may include, but is not limited to: determining the original motion vector as a central motion vector, and determining an edge motion vector corresponding to the central motion vector, wherein the edge motion vector is different from the central motion vector; then, obtaining the coding performance of the central motion vector according to the template of the current image block, and obtaining the coding performance of the edge motion vector according to the template of the current image block; then, a candidate motion vector corresponding to the original reference frame may be determined from the center motion vector and the edge motion vector according to the encoding performance of the center motion vector and the encoding performance of the edge motion vector.
In one example, obtaining the candidate motion vector corresponding to the candidate reference frame according to the initial motion vector may include, but is not limited to: a first candidate reference frame is any one of candidate reference frames, and aiming at the first candidate reference frame, an initial motion vector of the first candidate reference frame is determined as a central motion vector, and an edge motion vector corresponding to the central motion vector is determined, wherein the edge motion vector is different from the central motion vector; obtaining the coding performance of a central motion vector and the coding performance of an edge motion vector according to a template of a current image block; and determining a candidate motion vector corresponding to the first candidate reference frame from the central motion vector and the edge motion vector according to the coding performance of the central motion vector and the coding performance of the edge motion vector.
For a detailed implementation flow of the fifteenth embodiment, reference may be made to the seventh embodiment and the eighth embodiment, but only the execution main body is changed from the encoding end to the decoding end, and details are not repeated here.
Example sixteen:
in this embodiment, the original mode information may be an original predicted image block, and the target mode information may be a target predicted image block, based on which, the decoding end may obtain the original predicted image block corresponding to the current image block, based on a template of the current image block, obtain the target predicted image block of the current image block according to the original predicted image block, and decode the encoded bitstream according to the target predicted image block.
The method for obtaining the target prediction image block of the current image block according to the original prediction image block by the decoding end based on the template of the current image block may include, but is not limited to: the decoding end obtains the transformation coefficient of the original prediction image block according to the template of the current image block and the reference block corresponding to the template; and then, performing illumination compensation on the original prediction image block according to the transformation coefficient to obtain a target prediction image block of the current image block.
In one example, before the decoding end obtains the transform coefficient of the original prediction image block according to the template of the current image block and the reference block corresponding to the template, the decoding end may further obtain a motion vector of the current image block and a reference frame index corresponding to the motion vector; then, the decoding end can obtain the reference block corresponding to the template according to the motion vector of the current image block and the reference frame index.
In an example, the decoding end obtains the transform coefficients of the original predicted image block according to the template of the current image block and the reference block corresponding to the template, which may include but is not limited to: the decoding end can obtain a transformation coefficient according to the parameter information of each pixel of the template of the current image block and the parameter information of each pixel of the reference block corresponding to the template; the parameter information may include a luminance value of each pixel.
For a detailed implementation flow of the sixteenth embodiment, refer to the ninth embodiment and the tenth embodiment, except that the execution main body is changed from the encoding end to the decoding end, which is not repeated herein.
Example seventeen:
based on the same application concept as the method, an embodiment of the present application further provides a decoding apparatus applied to a decoding end, as shown in fig. 6, which is a structural diagram of the apparatus, and the apparatus includes:
an obtaining module 61, configured to obtain motion information of a candidate image block of a current image block; acquiring a template of the current image block according to the motion information of the candidate image block; a decoding module 62, configured to decode an encoded bitstream according to the template of the current image block.
The decoding module 62 is specifically configured to, when decoding the encoded bitstream according to the template of the current image block: acquiring original mode information of the current image block; acquiring target mode information of the current image block according to the original mode information based on the template of the current image block; decoding the encoded bitstream according to the target mode information.
The decoding apparatus further includes: a receiving module, configured to receive a coded bit stream from an encoding end, where the coded bit stream carries first indication information; the first indication information is used for indicating a template based on the current image block, and acquiring target mode information of the current image block according to the original mode information; the obtaining module 61 is further configured to obtain motion information of a candidate image block of the current image block according to the first indication information; and acquiring the template of the current image block according to the motion information of the candidate image block.
The receiving module is further configured to receive a coded bit stream from an encoding end, where the coded bit stream carries second indication information; wherein the second indication information is used to indicate that the encoded bitstream is decoded based on the original mode information; the decoding module 62 is further configured to obtain the original mode information according to the second indication information; decoding the encoded bitstream based on the original mode information.
The obtaining module 61 is further configured to obtain motion information of a candidate image block of the current image block according to locally preset first policy information; acquiring the template of the current image block according to the motion information of the candidate image block, wherein the first strategy information is used for indicating the template based on the current image block and acquiring target mode information of the current image block according to the original mode information; alternatively, the first and second electrodes may be,
the decoding module 62 is further configured to obtain the original mode information according to second policy information preset locally; decoding the encoded bitstream based on the original mode information, wherein the second policy information indicates that the encoded bitstream is decoded based on the original mode information; alternatively, the first and second electrodes may be,
the decoding module 62 is further configured to decode the encoded bitstream according to a locally preset third policy information; the third strategy information is the strategy information which is the same as the candidate image block of the current image block.
The original mode information is an original motion vector of the current image block, and the target mode information is a target motion vector of the current image block; the decoding module 62 is specifically configured to, based on the template of the current image block, when obtaining the target mode information of the current image block according to the original mode information: determining the original motion vector as a central motion vector; determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; and determining the target motion vector from the central motion vector and the edge motion vector according to the coding performance of the central motion vector and the coding performance of the edge motion vector.
The original mode information is an original motion vector and an original reference frame of the current image block, and the target mode information is a target motion vector and a target reference frame of the current image block; the decoding module 62 is specifically configured to, based on the template of the current image block, when obtaining the target mode information of the current image block according to the original mode information: based on the template of the current image block, obtaining a candidate motion vector corresponding to the original reference frame according to the original motion vector; acquiring initial motion vectors corresponding to the candidate reference frames according to the initial motion vectors; obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector; selecting a candidate motion vector with optimal coding performance from the candidate motion vectors corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames; and determining the candidate motion vector with the optimal coding performance as the target motion vector, and determining a reference frame corresponding to the candidate motion vector with the optimal coding performance as the target reference frame.
The decoding module 62 is specifically configured to, when obtaining the initial motion vector corresponding to each candidate reference frame according to the initial motion vector: and aiming at the first candidate reference frame, acquiring the initial motion vector of the first candidate reference frame according to the distance between the frame of the current image block and the original reference frame, the distance between the frame of the current image block and the first candidate reference frame and the original motion vector.
The decoding module 62 is specifically configured to, based on the template of the current image block, obtain a candidate motion vector corresponding to the original reference frame according to the original motion vector: determining the original motion vector as a central motion vector; determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; and determining a candidate motion vector corresponding to the original reference frame from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
The decoding module 62 is specifically configured to, when obtaining the candidate motion vector corresponding to the candidate reference frame according to the initial motion vector: a first candidate reference frame is any one of candidate reference frames, and an initial motion vector of the first candidate reference frame is determined as a central motion vector aiming at the first candidate reference frame; determining an edge motion vector corresponding to the center motion vector; the edge motion vector is different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; and determining the candidate motion vector corresponding to the first candidate reference frame from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
The original mode information is an original predicted image block, the target mode information is a target predicted image block, and the decoding module 62 is specifically configured to, based on the template of the current image block, obtain the target mode information of the current image block according to the original mode information: obtaining a transformation coefficient of the original prediction image block according to the template and a reference block corresponding to the template; and carrying out illumination compensation on the original prediction image block according to the transformation coefficient to obtain the target prediction image block.
The decoding module 62 is further configured to: acquiring a motion vector of a current image block and a reference frame index corresponding to the motion vector; and obtaining a reference block corresponding to the template according to the motion vector of the current image block and the reference frame index.
The decoding module 62 is specifically configured to, when obtaining the transform coefficient of the original prediction image block according to the template and the reference block corresponding to the template: acquiring the transformation coefficient according to the parameter information of each pixel of the template and the parameter information of each pixel of the reference block corresponding to the template; wherein the parameter information includes a luminance value.
In one example, the candidate image blocks include M first candidate image blocks and N second candidate image blocks, where M is a natural number greater than or equal to 1 and N is a natural number greater than or equal to 0, or M is a natural number greater than or equal to 0 and N is a natural number greater than or equal to 1; the first candidate image block is a candidate image block on the upper side of the current image block, and the second candidate image block is a candidate image block on the left side of the current image block; the obtaining module 61 is specifically configured to, when obtaining the template of the current image block according to the motion information of the candidate image block: determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks; determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks; determining the first template as the template of the current image block; or, determining the second template as the template of the current image block; or, determining the first template and the second template as the template of the current image block after splicing.
In one example, the first candidate image block includes a candidate image block on an upper side of the current image block and/or a sub-adjacent image block; the prediction mode of the candidate image block is an inter mode or an intra mode; the prediction mode of the secondary adjacent image block is an inter-frame mode; the second candidate image block comprises a candidate image block on the left side of the current image block and/or a secondary adjacent image block; the prediction mode of the candidate image block is an inter mode or an intra mode; and the prediction mode of the secondary adjacent image block is an inter-frame mode.
In an example, when M is greater than 1, the first template includes M sub-templates or P sub-templates, and is formed by splicing the M sub-templates or the P sub-templates, where P is the number of first candidate image blocks in the inter mode; and when M is equal to 1, the first template comprises a first sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the upper side of the current image block.
In an example, the motion information includes a motion vector and a reference frame index of the first candidate image block, and the obtaining module 61 is specifically configured to, when determining the first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks: for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block; and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as the ith sub-template included in the first template according to the determined reference image block.
The obtaining module 61, according to the motion vector prediction modes and the motion information of the M first candidate image blocks, is specifically configured to: for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as an ith sub-template included by the first template; or determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a first transverse length and a first longitudinal length as an ith sub-template included by the first template; and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
The first transverse length and the transverse length of the first candidate image block meet a first proportional relation, or the first transverse length and the transverse length of the current image block meet a second proportional relation, or the first transverse length and the transverse length of the current image block are equal to a first preset length;
the first longitudinal length and the longitudinal length of the first candidate image block meet a third proportional relation, or the first longitudinal length and the longitudinal length of the current image block meet a fourth proportional relation, or the first longitudinal length and the longitudinal length of the first candidate image block are equal to a second preset length.
In an example, when N is greater than 1, the second template includes N sub-templates or R sub-templates, and is formed by splicing the N sub-templates or the R sub-templates, where R is the number of second candidate image blocks in the inter mode; and when N is equal to 1, the second template comprises a second sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the left side of the current image block.
In an example, the motion information includes a motion vector and a reference frame index of the second candidate image block, and the obtaining module 61 is specifically configured to, when determining the second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks: for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block; and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as the ith sub-template included by the second template according to the determined reference image block.
The obtaining module 61, according to the motion vector prediction modes and the motion information of the N second candidate image blocks, is specifically configured to: for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as an ith sub-template included by the second template; or determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a second transverse length and a second longitudinal length as an ith sub-template included by the first template; and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
The second transverse length and the transverse length of the second candidate image block meet a fifth proportional relation, or the second transverse length and the transverse length of the current image block meet a sixth proportional relation, or the second transverse length and the transverse length of the current image block are equal to a third preset length;
the second vertical length and the vertical length of the second candidate image block satisfy a seventh proportional relationship, or satisfy an eighth proportional relationship with the vertical length of the current image block, or are equal to a fourth preset length.
The obtaining module 61 is specifically configured to, when obtaining the template of the current image block according to the motion information of the candidate image block: when the current image block corresponds to a plurality of pieces of motion information, obtaining a template corresponding to the motion information according to each piece of motion information; and acquiring a weight parameter corresponding to each piece of motion information, and acquiring a template of the current image block according to the weight parameter corresponding to each piece of motion information and the template corresponding to the motion information.
Based on the same application concept as the method, an embodiment of the present application further provides an encoding apparatus applied to an encoding end, as shown in fig. 7, which is a structural diagram of the apparatus, and the apparatus includes:
an obtaining module 71, configured to obtain motion information of a candidate image block of a current image block; acquiring a template of the current image block according to the motion information of the candidate image block; the encoding module 72 is configured to encode a bitstream according to the template of the current image block to obtain an encoded bitstream; a sending module 73, configured to send the encoded bit stream to a decoding end.
The encoding module 72 encodes a bitstream according to the template of the current image block, and is specifically configured to: acquiring original mode information of the current image block; acquiring target mode information of the current image block according to the original mode information based on the template of the current image block; and coding the bit stream according to the target mode information to obtain a coded bit stream.
When encoding a bitstream according to the target mode information, the sending module 73 sends, to a decoding end, an encoded bitstream carrying first indication information, where the first indication information is used to indicate a template based on the current image block, and obtains the target mode information of the current image block according to the original mode information.
When the bitstream is encoded according to the original mode information, the sending module 73 sends, to a decoding end, a coded bitstream carrying second indication information, where the second indication information is used to indicate that the coded bitstream is decoded based on the original mode information.
The original mode information is an original motion vector of the current image block, and the target mode information is a target motion vector of the current image block; the encoding module 72 is specifically configured to, based on the template of the current image block, obtain target mode information of the current image block according to the original mode information: determining the original motion vector as a central motion vector; determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; and determining the target motion vector from the central motion vector and the edge motion vector according to the coding performance of the central motion vector and the coding performance of the edge motion vector.
The original mode information is an original motion vector and an original reference frame of the current image block, and the target mode information is a target motion vector and a target reference frame of the current image block; the encoding module 72 is specifically configured to, based on the template of the current image block, obtain target mode information of the current image block according to the original mode information: based on the template of the current image block, obtaining a candidate motion vector corresponding to the original reference frame according to the original motion vector; acquiring initial motion vectors corresponding to the candidate reference frames according to the initial motion vectors; obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector; selecting a candidate motion vector with optimal coding performance from the candidate motion vectors corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames; and determining the candidate motion vector with the optimal coding performance as the target motion vector, and determining a reference frame corresponding to the candidate motion vector with the optimal coding performance as the target reference frame.
The encoding module 72 is specifically configured to, when obtaining the initial motion vector corresponding to each candidate reference frame according to the initial motion vector: and aiming at the first candidate reference frame, acquiring the initial motion vector of the first candidate reference frame according to the distance between the frame of the current image block and the original reference frame, the distance between the frame of the current image block and the first candidate reference frame and the original motion vector.
The encoding module 72 is specifically configured to, based on the template of the current image block, obtain a candidate motion vector corresponding to the original reference frame according to the original motion vector: determining the original motion vector as a central motion vector; determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; and determining a candidate motion vector corresponding to the original reference frame from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
The encoding module 72 is specifically configured to, when obtaining the candidate motion vector corresponding to the candidate reference frame according to the initial motion vector: a first candidate reference frame is any one of candidate reference frames, and an initial motion vector of the first candidate reference frame is determined as a central motion vector aiming at the first candidate reference frame; determining an edge motion vector corresponding to the center motion vector; the edge motion vector is different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; and determining the candidate motion vector corresponding to the first candidate reference frame from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
The original mode information is an original predicted image block, the target mode information is a target predicted image block, and the encoding module 72 is specifically configured to, based on the template of the current image block, obtain the target mode information of the current image block according to the original mode information: obtaining a transformation coefficient of the original prediction image block according to the template and a reference block corresponding to the template; and carrying out illumination compensation on the original prediction image block according to the transformation coefficient to obtain the target prediction image block.
The encoding module 72 is further configured to: acquiring a motion vector of a current image block and a reference frame index corresponding to the motion vector; and obtaining a reference block corresponding to the template according to the motion vector of the current image block and the reference frame index.
The encoding module 72 is specifically configured to, when obtaining the transform coefficient of the original prediction image block according to the template and the reference block corresponding to the template: acquiring the transformation coefficient according to the parameter information of each pixel of the template and the parameter information of each pixel of the reference block corresponding to the template; wherein the parameter information includes a luminance value.
The candidate image blocks comprise M first candidate image blocks and N second candidate image blocks, wherein M is a natural number which is greater than or equal to 1, N is a natural number which is greater than or equal to 0, or M is a natural number which is greater than or equal to 0, and N is a natural number which is greater than or equal to 1; the first candidate image block is a candidate image block on the upper side of the current image block, and the second candidate image block is a candidate image block on the left side of the current image block; the obtaining module 71 is specifically configured to, when obtaining the template of the current image block according to the motion information of the candidate image block: determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks; determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks; determining the first template as the template of the current image block; or, determining the second template as the template of the current image block; or, determining the first template and the second template as the template of the current image block after splicing.
In one example, the first candidate image block includes a candidate image block on an upper side of the current image block and/or a sub-adjacent image block; the prediction mode of the candidate image block is an inter mode or an intra mode; the prediction mode of the secondary adjacent image block is an inter-frame mode; the second candidate image block comprises a candidate image block on the left side of the current image block and/or a secondary adjacent image block; the prediction mode of the candidate image block is an inter mode or an intra mode; and the prediction mode of the secondary adjacent image block is an inter-frame mode.
In an example, when M is greater than 1, the first template includes M sub-templates or P sub-templates, and is formed by splicing the M sub-templates or the P sub-templates, where P is the number of first candidate image blocks in the inter mode; and when M is equal to 1, the first template comprises a first sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the upper side of the current image block.
The motion information includes a motion vector and a reference frame index of the first candidate image block, and the obtaining module 71 is specifically configured to, when determining the first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks: for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block; and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as the ith sub-template included in the first template according to the determined reference image block.
The obtaining module 71, according to the motion vector prediction modes and the motion information of the M first candidate image blocks, is specifically configured to: for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as an ith sub-template included by the first template; or determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a first transverse length and a first longitudinal length as an ith sub-template included by the first template; and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
The first transverse length and the transverse length of the first candidate image block meet a first proportional relation, or the first transverse length and the transverse length of the current image block meet a second proportional relation, or the first transverse length and the transverse length of the current image block are equal to a first preset length;
the first longitudinal length and the longitudinal length of the first candidate image block meet a third proportional relation, or the first longitudinal length and the longitudinal length of the current image block meet a fourth proportional relation, or the first longitudinal length and the longitudinal length of the first candidate image block are equal to a second preset length.
In an example, when N is greater than 1, the second template includes N sub-templates or R sub-templates, and is formed by splicing the N sub-templates or the R sub-templates, where R is the number of second candidate image blocks in the inter mode; and when N is equal to 1, the second template comprises a second sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the left side of the current image block.
The motion information includes a motion vector and a reference frame index of the second candidate image block, and the obtaining module 71 is specifically configured to, when determining the second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks: for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block; and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as the ith sub-template included by the second template according to the determined reference image block.
The obtaining module 71, according to the motion vector prediction modes and the motion information of the N second candidate image blocks, is specifically configured to: for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as an ith sub-template included by the second template; or determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a second transverse length and a second longitudinal length as an ith sub-template included by the first template; and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
The second transverse length and the transverse length of the second candidate image block meet a fifth proportional relation, or the second transverse length and the transverse length of the current image block meet a sixth proportional relation, or the second transverse length and the transverse length of the current image block are equal to a third preset length;
the second vertical length and the vertical length of the second candidate image block satisfy a seventh proportional relationship, or satisfy an eighth proportional relationship with the vertical length of the current image block, or are equal to a fourth preset length.
The obtaining module 71 is specifically configured to, when obtaining the template of the current image block according to the motion information of the candidate image block: when the current image block corresponds to a plurality of pieces of motion information, obtaining a template corresponding to the motion information according to each piece of motion information; and acquiring a weight parameter corresponding to each piece of motion information, and acquiring a template of the current image block according to the weight parameter corresponding to each piece of motion information and the template corresponding to the motion information.
In terms of hardware, the hardware architecture diagram of the decoding-side device provided in the embodiment of the present application may specifically refer to fig. 8. The method comprises the following steps: a processor 81 and a machine-readable storage medium 82, wherein: the machine-readable storage medium 82 stores machine-executable instructions executable by the processor 81; the processor 81 is configured to execute machine executable instructions to implement the decoding method disclosed in the above example of the present application.
Based on the same application concept as the method, embodiments of the present application further provide a machine-readable storage medium, where several computer instructions are stored, and when the computer instructions are executed by a processor, the decoding method disclosed in the above example of the present application can be implemented.
In terms of hardware, the hardware architecture diagram of the encoding end device provided in the embodiment of the present application may specifically refer to fig. 9. The method comprises the following steps: a processor 91 and a machine-readable storage medium 92, wherein: the machine-readable storage medium 92 stores machine-executable instructions executable by the processor 91; the processor 91 is configured to execute machine-executable instructions to implement the encoding methods disclosed in the above examples of the present application.
Based on the same application concept as the method, embodiments of the present application further provide a machine-readable storage medium, where several computer instructions are stored, and when the computer instructions are executed by a processor, the encoding method disclosed in the above example of the present application can be implemented.
The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: a RAM (random Access Memory), a volatile Memory, a non-volatile Memory, a flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., an optical disk, a dvd, etc.), or similar storage medium, or a combination thereof.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Furthermore, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (45)

1. A decoding method applied to a decoding end, the method comprising:
acquiring motion information of candidate image blocks of a current image block;
acquiring a template of the current image block according to the motion information of the candidate image block;
decoding an encoded bitstream according to the template of the current image block;
wherein said decoding an encoded bitstream according to said template of said current image block comprises: acquiring original mode information of the current image block; acquiring target mode information of the current image block according to the original mode information based on the template of the current image block; decoding the encoded bitstream according to the target mode information;
the original mode information is an original motion vector of the current image block, and the target mode information is a target motion vector of the current image block; the obtaining, by the template based on the current image block and according to the original mode information, target mode information of the current image block specifically includes: determining the original motion vector as a central motion vector; determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector; obtaining a cost value of the central motion vector and a cost value of the edge motion vector according to the template; determining the target motion vector from the central motion vector and the edge motion vector according to the cost value of the central motion vector and the cost value of the edge motion vector;
wherein the determining the target motion vector from the center motion vector and the edge motion vector according to the cost value of the center motion vector and the cost value of the edge motion vector comprises: and selecting the motion vector with the minimum cost value from the central motion vector and the edge motion vector as the target motion vector.
2. The method of claim 1,
before obtaining the target mode information of the current image block according to the original mode information based on the template of the current image block, the method further includes:
receiving a coded bit stream from a coding end, wherein the coded bit stream carries first indication information; the first indication information is used for indicating a template based on the current image block, and acquiring target mode information of the current image block according to the original mode information;
acquiring motion information of candidate image blocks of the current image block according to the first indication information; and acquiring the template of the current image block according to the motion information of the candidate image block.
3. The method of claim 1,
before obtaining the target mode information of the current image block according to the original mode information based on the template of the current image block, the method further includes:
receiving a coded bit stream from a coding end, wherein the coded bit stream carries second indication information; wherein the second indication information is used to indicate that the encoded bitstream is decoded based on the original mode information;
and acquiring the original mode information according to the second indication information, and decoding the coded bit stream based on the original mode information.
4. The method of claim 1,
before obtaining the target mode information of the current image block according to the original mode information based on the template of the current image block, the method further includes:
acquiring motion information of candidate image blocks of a current image block according to locally preset first strategy information; acquiring the template of the current image block according to the motion information of the candidate image block, wherein the first strategy information is used for indicating the template based on the current image block and acquiring target mode information of the current image block according to the original mode information; alternatively, the first and second electrodes may be,
acquiring original mode information according to second strategy information preset locally, and decoding the coded bit stream based on the original mode information, wherein the second strategy information is used for indicating that the coded bit stream is decoded based on the original mode information; alternatively, the first and second electrodes may be,
decoding the coded bit stream according to locally preset third strategy information; and the third strategy information is the strategy information which is the same as the candidate image block of the current image block.
5. The method according to claim 1, wherein the original mode information is an original motion vector and an original reference frame of the current image block, and the target mode information is a target motion vector and a target reference frame of the current image block; the obtaining, by the template based on the current image block and according to the original mode information, target mode information of the current image block specifically includes:
based on the template of the current image block, obtaining a candidate motion vector corresponding to the original reference frame according to the original motion vector;
acquiring initial motion vectors corresponding to the candidate reference frames according to the initial motion vectors;
obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector;
selecting a candidate motion vector with the optimal cost value from the candidate motion vectors corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames;
and determining the candidate motion vector with the optimal cost value as the target motion vector, and determining a reference frame corresponding to the candidate motion vector with the optimal cost value as the target reference frame.
6. The method of claim 5,
obtaining an initial motion vector corresponding to each candidate reference frame according to the initial motion vector, including:
and aiming at the first candidate reference frame, acquiring the initial motion vector of the first candidate reference frame according to the distance between the frame of the current image block and the original reference frame, the distance between the frame of the current image block and the first candidate reference frame and the original motion vector.
7. The method of claim 5, wherein obtaining the candidate motion vector corresponding to the original reference frame according to the original motion vector based on the template of the current image block comprises:
determining the original motion vector as a central motion vector;
determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector;
obtaining a cost value of the central motion vector and a cost value of the edge motion vector according to the template;
and determining a candidate motion vector corresponding to the original reference frame from the central motion vector and the edge motion vector according to the cost value of the central motion vector and the cost value of the edge motion vector.
8. The method according to claim 5 or 6,
obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector, including:
a first candidate reference frame, for which any one of the candidate reference frames is selected,
determining an initial motion vector of the first candidate reference frame as a central motion vector;
determining an edge motion vector corresponding to the center motion vector; the edge motion vector is different from the center motion vector;
obtaining a cost value of the central motion vector and a cost value of the edge motion vector according to the template;
and determining the candidate motion vector corresponding to the first candidate reference frame from the central motion vector and the edge motion vector according to the cost value of the central motion vector and the cost value of the edge motion vector.
9. The method of claim 1,
the original mode information is an original prediction image block, the target mode information is a target prediction image block, the template based on the current image block obtains the target mode information of the current image block according to the original mode information, and the method specifically includes:
obtaining a transformation coefficient of the original prediction image block according to the template and a reference block corresponding to the template;
and carrying out illumination compensation on the original prediction image block according to the transformation coefficient to obtain the target prediction image block.
10. The method according to claim 9, wherein before obtaining the transform coefficients of the original predicted image block according to the template and the reference block corresponding to the template, the method further comprises:
acquiring a motion vector of a current image block and a reference frame index corresponding to the motion vector;
and obtaining a reference block corresponding to the template according to the motion vector of the current image block and the reference frame index.
11. The method according to claim 9, wherein said obtaining transform coefficients of said original predicted image block according to said template and a reference block corresponding to said template comprises:
acquiring the transformation coefficient according to the parameter information of each pixel of the template and the parameter information of each pixel of the reference block corresponding to the template; wherein the parameter information includes a luminance value.
12. The method according to claim 1, wherein the candidate image blocks comprise M first candidate image blocks and N second candidate image blocks, M being a natural number greater than or equal to 1 and N being a natural number greater than or equal to 0, or M being a natural number greater than or equal to 0 and N being a natural number greater than or equal to 1; the first candidate image block is a candidate image block on the upper side of the current image block, and the second candidate image block is a candidate image block on the left side of the current image block;
the obtaining the template of the current image block according to the motion information of the candidate image block includes:
determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks;
determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks;
determining the first template as the template of the current image block; alternatively, the first and second electrodes may be,
determining the second template as the template of the current image block; alternatively, the first and second electrodes may be,
and determining the first template and the second template as the template of the current image block after splicing.
13. The method of claim 12,
the first candidate image block comprises a candidate image block and/or a secondary adjacent image block on the upper side of the current image block, the prediction mode of the candidate image block is an inter-frame mode or an intra-frame mode, and the prediction mode of the secondary adjacent image block is an inter-frame mode;
the second candidate image block comprises a candidate image block on the left side of the current image block and/or a secondary adjacent image block, the prediction mode of the candidate image block is an inter mode or an intra mode, and the prediction mode of the secondary adjacent image block is an inter mode.
14. The method according to claim 12, wherein when M is greater than 1, the first template includes M sub-templates or P sub-templates, and is formed by splicing the M sub-templates or the P sub-templates, where P is the number of first candidate image blocks in the inter mode;
and when M is equal to 1, the first template comprises a first sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the upper side of the current image block.
15. The method according to claim 14, wherein the motion information comprises a motion vector and a reference frame index of the first candidate image block, and the determining the first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks comprises:
for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index;
determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block;
and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as the ith sub-template included in the first template according to the determined reference image block.
16. The method as claimed in claim 14, wherein determining the first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks comprises:
for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as an ith sub-template included by the first template; alternatively, the first and second electrodes may be,
determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a first transverse length and a first longitudinal length as an ith sub-template included by the first template;
and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
17. The method according to claim 15 or 16,
the first transverse length and the transverse length of the first candidate image block meet a first proportional relation, or the first transverse length and the transverse length of the current image block meet a second proportional relation, or the first transverse length and the transverse length of the current image block are equal to a first preset length;
the first longitudinal length and the longitudinal length of the first candidate image block meet a third proportional relation, or the first longitudinal length and the longitudinal length of the current image block meet a fourth proportional relation, or the first longitudinal length and the longitudinal length of the first candidate image block are equal to a second preset length.
18. The method according to claim 12, wherein when N is greater than 1, the second template includes N sub-templates or R sub-templates, and is formed by splicing the N sub-templates or the R sub-templates, where R is the number of second candidate image blocks in the inter mode;
and when N is equal to 1, the second template comprises a second sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the left side of the current image block.
19. The method of claim 18, wherein the motion information comprises a motion vector and a reference frame index of the second candidate image block, and wherein determining the second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks comprises:
for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index;
determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block;
and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as the ith sub-template included by the second template according to the determined reference image block.
20. The method as claimed in claim 18, wherein determining the second template according to the motion vector prediction mode and the motion information of the N second candidate image blocks comprises:
for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as an ith sub-template included by the second template; alternatively, the first and second electrodes may be,
determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a second transverse length and a second longitudinal length as an ith sub-template included by the first template;
and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
21. The method of claim 19 or 20,
the second transverse length and the transverse length of the second candidate image block meet a fifth proportional relation, or the second transverse length and the transverse length of the current image block meet a sixth proportional relation, or the second transverse length and the transverse length of the current image block are equal to a third preset length;
the second vertical length and the vertical length of the second candidate image block satisfy a seventh proportional relationship, or satisfy an eighth proportional relationship with the vertical length of the current image block, or are equal to a fourth preset length.
22. The method of claim 12,
the obtaining the template of the current image block according to the motion information of the candidate image block includes:
when the current image block corresponds to a plurality of pieces of motion information, obtaining a template corresponding to the motion information according to each piece of motion information;
and acquiring a weight parameter corresponding to each piece of motion information, and acquiring a template of the current image block according to the weight parameter corresponding to each piece of motion information and the template corresponding to the motion information.
23. An encoding method applied to an encoding end, the method comprising:
acquiring motion information of candidate image blocks of a current image block;
acquiring a template of the current image block according to the motion information of the candidate image block;
coding a bit stream according to the template of the current image block to obtain a coded bit stream;
transmitting the coded bit stream to a decoding end;
wherein the encoding a bitstream according to the template of the current image block to obtain an encoded bitstream includes: acquiring original mode information of the current image block; acquiring target mode information of the current image block according to the original mode information based on the template of the current image block; coding a bit stream according to the target mode information to obtain a coded bit stream;
the original mode information is an original motion vector of the current image block, and the target mode information is a target motion vector of the current image block; the obtaining, by the template based on the current image block and according to the original mode information, target mode information of the current image block specifically includes: determining the original motion vector as a central motion vector; determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector; obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template; determining the target motion vector from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector;
wherein the determining the target motion vector from the center motion vector and the edge motion vector according to the encoding performance of the center motion vector and the encoding performance of the edge motion vector comprises: and selecting the motion vector with the maximum coding performance from the central motion vector and the edge motion vector as the target motion vector.
24. The method of claim 23, further comprising:
when the bit stream is coded according to the target mode information, sending the coded bit stream carrying first indication information to a decoding end, wherein the first indication information is used for indicating a template based on the current image block, and acquiring the target mode information of the current image block according to the original mode information.
25. The method of claim 23, further comprising:
and when the bit stream is coded according to the original mode information, sending the coded bit stream carrying second indication information to a decoding end, wherein the second indication information is used for indicating that the coded bit stream is decoded based on the original mode information.
26. The method according to claim 23, wherein the original mode information is an original motion vector and an original reference frame of the current image block, and the target mode information is a target motion vector and a target reference frame of the current image block; the obtaining, by the template based on the current image block and according to the original mode information, target mode information of the current image block specifically includes:
based on the template of the current image block, obtaining a candidate motion vector corresponding to the original reference frame according to the original motion vector;
acquiring initial motion vectors corresponding to the candidate reference frames according to the initial motion vectors;
obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector;
selecting a candidate motion vector with optimal coding performance from the candidate motion vectors corresponding to the original reference frame and the candidate motion vectors corresponding to the candidate reference frames;
and determining the candidate motion vector with the optimal coding performance as the target motion vector, and determining a reference frame corresponding to the candidate motion vector with the optimal coding performance as the target reference frame.
27. The method of claim 26,
obtaining an initial motion vector corresponding to each candidate reference frame according to the initial motion vector, including:
and aiming at the first candidate reference frame, acquiring the initial motion vector of the first candidate reference frame according to the distance between the frame of the current image block and the original reference frame, the distance between the frame of the current image block and the first candidate reference frame and the original motion vector.
28. The method of claim 26,
based on the template of the current image block, obtaining a candidate motion vector corresponding to the original reference frame according to the original motion vector, including:
determining the original motion vector as a central motion vector;
determining an edge motion vector corresponding to the center motion vector, the edge motion vector being different from the center motion vector;
obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template;
and determining a candidate motion vector corresponding to the original reference frame from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
29. The method of claim 26 or 27,
obtaining a candidate motion vector corresponding to the candidate reference frame according to the initial motion vector, including:
a first candidate reference frame, for which any one of the candidate reference frames is selected,
determining an initial motion vector of the first candidate reference frame as a central motion vector;
determining an edge motion vector corresponding to the center motion vector; the edge motion vector is different from the center motion vector;
obtaining the coding performance of the central motion vector and the coding performance of the edge motion vector according to the template;
and determining the candidate motion vector corresponding to the first candidate reference frame from the center motion vector and the edge motion vector according to the coding performance of the center motion vector and the coding performance of the edge motion vector.
30. The method of claim 23,
the original mode information is an original prediction image block, the target mode information is a target prediction image block, the template based on the current image block obtains the target mode information of the current image block according to the original mode information, and the method specifically includes:
obtaining a transformation coefficient of the original prediction image block according to the template and a reference block corresponding to the template;
and carrying out illumination compensation on the original prediction image block according to the transformation coefficient to obtain the target prediction image block.
31. The method according to claim 30, wherein before obtaining transform coefficients of said original predicted image block according to said template and a reference block corresponding to said template, further comprising:
acquiring a motion vector of a current image block and a reference frame index corresponding to the motion vector;
and obtaining a reference block corresponding to the template according to the motion vector of the current image block and the reference frame index.
32. The method according to claim 30, wherein said obtaining transform coefficients of said original predicted image block according to said template and a reference block corresponding to said template comprises:
acquiring the transformation coefficient according to the parameter information of each pixel of the template and the parameter information of each pixel of the reference block corresponding to the template; wherein the parameter information includes a luminance value.
33. The method according to claim 23, wherein the candidate image blocks comprise M first candidate image blocks and N second candidate image blocks, M being a natural number greater than or equal to 1 and N being a natural number greater than or equal to 0, or M being a natural number greater than or equal to 0 and N being a natural number greater than or equal to 1; the first candidate image block is a candidate image block on the upper side of the current image block, and the second candidate image block is a candidate image block on the left side of the current image block;
the obtaining the template of the current image block according to the motion information of the candidate image block includes:
determining a first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks;
determining a second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks;
determining the first template as the template of the current image block; alternatively, the first and second electrodes may be,
determining the second template as the template of the current image block; alternatively, the first and second electrodes may be,
and determining the first template and the second template as the template of the current image block after splicing.
34. The method of claim 33,
the first candidate image block comprises a candidate image block and/or a secondary adjacent image block on the upper side of the current image block, the prediction mode of the candidate image block is an inter-frame mode or an intra-frame mode, and the prediction mode of the secondary adjacent image block is an inter-frame mode;
the second candidate image block comprises a candidate image block on the left side of the current image block and/or a secondary adjacent image block, the prediction mode of the candidate image block is an inter mode or an intra mode, and the prediction mode of the secondary adjacent image block is an inter mode.
35. The method according to claim 33, wherein when M is greater than 1, the first template includes M sub-templates or P sub-templates, and is formed by splicing the M sub-templates or the P sub-templates, where P is the number of first candidate image blocks in the inter mode;
and when M is equal to 1, the first template comprises a first sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the upper side of the current image block.
36. The method of claim 35, wherein the motion information comprises a motion vector and a reference frame index of the first candidate image block, and wherein determining the first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks comprises:
for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index;
determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block;
and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as the ith sub-template included in the first template according to the determined reference image block.
37. The method as claimed in claim 35, wherein determining the first template according to the motion vector prediction modes and the motion information of the M first candidate image blocks comprises:
for an ith candidate image block in the M first candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a first transverse length and a first longitudinal length as an ith sub-template included by the first template; alternatively, the first and second electrodes may be,
determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a first transverse length and a first longitudinal length as an ith sub-template included by the first template;
and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
38. The method of claim 36 or 37,
the first transverse length and the transverse length of the first candidate image block meet a first proportional relation, or the first transverse length and the transverse length of the current image block meet a second proportional relation, or the first transverse length and the transverse length of the current image block are equal to a first preset length;
the first longitudinal length and the longitudinal length of the first candidate image block meet a third proportional relation, or the first longitudinal length and the longitudinal length of the current image block meet a fourth proportional relation, or the first longitudinal length and the longitudinal length of the first candidate image block are equal to a second preset length.
39. The method according to claim 33, wherein when N is greater than 1, the second template comprises N sub-templates or R sub-templates, and is formed by splicing the N sub-templates or the R sub-templates, where R is the number of second candidate image blocks in the inter mode;
and when N is equal to 1, the second template comprises a second sub-template which is determined according to the motion vector prediction mode and the motion information of the first candidate image block on the left side of the current image block.
40. The method of claim 39, wherein the motion information comprises a motion vector and a reference frame index of the second candidate image block, and wherein determining the second template according to the motion vector prediction modes and the motion information of the N second candidate image blocks comprises:
for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an inter mode, determining a reference frame image corresponding to the ith candidate image block according to the reference frame index;
determining a reference image block corresponding to the ith candidate image block from the reference frame image according to the motion vector of the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector of the ith candidate image block;
and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as the ith sub-template included by the second template according to the determined reference image block.
41. The method as claimed in claim 39, wherein determining the second template according to the motion vector prediction mode and the motion information of the N second candidate image blocks comprises:
for an ith candidate image block in the N second candidate image blocks, when the motion vector prediction mode of the ith candidate image block is determined to be an intra-frame mode, filling the ith candidate image block according to a default value, and acquiring image blocks with the sizes of a second transverse length and a second longitudinal length as an ith sub-template included by the second template; alternatively, the first and second electrodes may be,
determining a reference frame image corresponding to the ith candidate image block according to the reference frame index corresponding to the ith candidate image block; determining a reference image block corresponding to the ith candidate image block from the reference frame image according to a motion vector corresponding to the ith candidate image block, wherein the relative position offset of the reference image block and the ith candidate image block is matched with the motion vector corresponding to the ith candidate image block; according to the determined reference image block, acquiring an image block with the size of a second transverse length and a second longitudinal length as an ith sub-template included by the first template;
and the reference frame index and the motion vector corresponding to the ith candidate image block are the reference frame index and the motion vector of the candidate image block of the ith candidate image block.
42. The method of claim 40 or 41,
the second transverse length and the transverse length of the second candidate image block meet a fifth proportional relation, or the second transverse length and the transverse length of the current image block meet a sixth proportional relation, or the second transverse length and the transverse length of the current image block are equal to a third preset length;
the second vertical length and the vertical length of the second candidate image block satisfy a seventh proportional relationship, or satisfy an eighth proportional relationship with the vertical length of the current image block, or are equal to a fourth preset length.
43. The method of claim 33,
the obtaining the template of the current image block according to the motion information of the candidate image block includes:
when the current image block corresponds to a plurality of pieces of motion information, obtaining a template corresponding to the motion information according to each piece of motion information;
and acquiring a weight parameter corresponding to each piece of motion information, and acquiring a template of the current image block according to the weight parameter corresponding to each piece of motion information and the template corresponding to the motion information.
44. A decoding-side apparatus, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute machine executable instructions to perform the method steps of any of claims 1-22.
45. An encoding side device, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute machine executable instructions to perform the method steps of any of claims 23-43.
CN201810692041.XA 2018-06-28 2018-06-28 Decoding and encoding method and device thereof Active CN110662033B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201810692041.XA CN110662033B (en) 2018-06-28 2018-06-28 Decoding and encoding method and device thereof
PCT/CN2019/093574 WO2020001591A1 (en) 2018-06-28 2019-06-28 Decoding and encoding methods and apparatus thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810692041.XA CN110662033B (en) 2018-06-28 2018-06-28 Decoding and encoding method and device thereof

Publications (2)

Publication Number Publication Date
CN110662033A CN110662033A (en) 2020-01-07
CN110662033B true CN110662033B (en) 2021-11-23

Family

ID=68984489

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810692041.XA Active CN110662033B (en) 2018-06-28 2018-06-28 Decoding and encoding method and device thereof

Country Status (2)

Country Link
CN (1) CN110662033B (en)
WO (1) WO2020001591A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117981301A (en) * 2021-09-27 2024-05-03 Oppo广东移动通信有限公司 Method for intra prediction, encoder, decoder and encoding/decoding system
CN116074529A (en) * 2022-01-07 2023-05-05 杭州海康威视数字技术股份有限公司 Image encoding and decoding method, device and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1497981A (en) * 2002-07-16 2004-05-19 ���ǵ�����ʽ���� Method and device for code and decode motion vector
WO2011071514A2 (en) * 2009-12-08 2011-06-16 Thomson Licensing Methods and apparatus for adaptive residual updating of template matching prediction for video encoding and decoding
CN102177716A (en) * 2008-10-09 2011-09-07 株式会社Ntt都科摩 Moving image encoding device, moving image decoding device, moving image encoding method, moving image decoding method, moving image encoding program, moving image decoding program, moving image processing system and moving image processing method
CN102215386A (en) * 2010-04-09 2011-10-12 华为技术有限公司 Video image block processing method and device
CN104980760A (en) * 2010-07-09 2015-10-14 三星电子株式会社 Method and apparatus for encoding and decoding motion vector
CN106604035A (en) * 2017-01-22 2017-04-26 北京君泊网络科技有限责任公司 Motion estimation method for video encoding and compression
WO2018065397A2 (en) * 2016-10-03 2018-04-12 Thomson Licensing Method and apparatus for encoding and decoding motion information
CN108134939A (en) * 2016-12-01 2018-06-08 北京金山云网络技术有限公司 A kind of method for estimating and device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5818969A (en) * 1995-05-12 1998-10-06 Intel Corporation Intelligent start for motion estimation search
KR100928325B1 (en) * 2007-10-15 2009-11-25 세종대학교산학협력단 Image encoding and decoding method and apparatus
CN101895675B (en) * 2010-07-26 2012-10-03 杭州海康威视软件有限公司 Motion detection method and device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1497981A (en) * 2002-07-16 2004-05-19 ���ǵ�����ʽ���� Method and device for code and decode motion vector
CN102177716A (en) * 2008-10-09 2011-09-07 株式会社Ntt都科摩 Moving image encoding device, moving image decoding device, moving image encoding method, moving image decoding method, moving image encoding program, moving image decoding program, moving image processing system and moving image processing method
WO2011071514A2 (en) * 2009-12-08 2011-06-16 Thomson Licensing Methods and apparatus for adaptive residual updating of template matching prediction for video encoding and decoding
CN102215386A (en) * 2010-04-09 2011-10-12 华为技术有限公司 Video image block processing method and device
CN104980760A (en) * 2010-07-09 2015-10-14 三星电子株式会社 Method and apparatus for encoding and decoding motion vector
WO2018065397A2 (en) * 2016-10-03 2018-04-12 Thomson Licensing Method and apparatus for encoding and decoding motion information
CN108134939A (en) * 2016-12-01 2018-06-08 北京金山云网络技术有限公司 A kind of method for estimating and device
CN106604035A (en) * 2017-01-22 2017-04-26 北京君泊网络科技有限责任公司 Motion estimation method for video encoding and compression

Also Published As

Publication number Publication date
CN110662033A (en) 2020-01-07
WO2020001591A1 (en) 2020-01-02

Similar Documents

Publication Publication Date Title
JP6545318B2 (en) Decryption method, apparatus, program, storage medium
CN110933426B (en) Decoding and encoding method and device thereof
US20120027092A1 (en) Image processing device, system and method
JP2010016454A (en) Image encoding apparatus and method, image decoding apparatus and method, and program
CN111698500B (en) Encoding and decoding method, device and equipment
CN111263144B (en) Motion information determination method and equipment
CN110662033B (en) Decoding and encoding method and device thereof
CN110662074B (en) Motion vector determination method and device
CN113873257B (en) Method, device and equipment for constructing motion information candidate list
CN113709457B (en) Decoding and encoding method, device and equipment
CN112449180B (en) Encoding and decoding method, device and equipment
CN110691247B (en) Decoding and encoding method and device
WO2010061515A1 (en) Dynamic image encoding device, encoding method, dynamic image decoding device, and decoding method
CN110365987B (en) Motion vector determination method, device and equipment
CN113422951B (en) Decoding and encoding method, device and equipment
CN113709486B (en) Encoding and decoding method, device and equipment
CN112055220B (en) Encoding and decoding method, device and equipment
KR101525325B1 (en) Intra prediction mode determination method and apparatus
CN113132734A (en) Encoding and decoding method, device and equipment
KR20120008271A (en) Methods and apparatus for the predicted motion vector selection using matching with neighboring pixels
KR20140087137A (en) Method and apparratus of adaptive bi-prediction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant