WO2025190011A1 - 基于模板匹配的预测方法、装置、设备及存储介质 - Google Patents

基于模板匹配的预测方法、装置、设备及存储介质

Info

Publication number
WO2025190011A1
WO2025190011A1 PCT/CN2025/076891 CN2025076891W WO2025190011A1 WO 2025190011 A1 WO2025190011 A1 WO 2025190011A1 CN 2025076891 W CN2025076891 W CN 2025076891W WO 2025190011 A1 WO2025190011 A1 WO 2025190011A1
Authority
WO
WIPO (PCT)
Prior art keywords
vector
search point
template
prediction
current block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/076891
Other languages
English (en)
French (fr)
Inventor
王力强
许晓中
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology (Shenzhen) Co Ltd
Original Assignee
Tencent Technology (Shenzhen) Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology (Shenzhen) Co Ltd filed Critical Tencent Technology (Shenzhen) Co Ltd
Priority to KR1020267019607A priority Critical patent/KR20260113278A/ko
Publication of WO2025190011A1 publication Critical patent/WO2025190011A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria

Definitions

  • the present application relates to the field of coding and decoding technology, and more specifically, to a prediction method, apparatus, device and storage medium based on template matching.
  • the inter-frame prediction mode or intra-frame block copy mode has searched for the image block closest to the current coding block in the reference area through template matching, and then encoded the vector information (for example, the motion vector used to indicate the position of the prediction block) and sent it to the decoding end, so that the decoding end can obtain the prediction block of the current block in the reference area through the vector information.
  • the vector information for example, the motion vector used to indicate the position of the prediction block
  • the embodiments of the present application provide a prediction method, apparatus, device, and storage medium based on template matching, which can improve the bit rate.
  • an embodiment of the present application provides a prediction method based on template matching, comprising:
  • a prediction block of the current block is determined.
  • an embodiment of the present application provides a prediction device based on template matching, comprising:
  • a first determining unit is configured to determine a first starting search point based on a first vector included in a first vector prediction VP in a vector prediction VP list of a current block;
  • a searching unit configured to search for a second vector of the current block in a reference area of the current block based on the first starting search point and using a template formed by reconstructed pixels around the current block;
  • the second determining unit is configured to determine a prediction block of the current block based on the second vector.
  • an electronic device including:
  • a processor adapted to implement computer instructions
  • a computer-readable storage medium stores computer instructions, wherein the computer instructions are suitable for being loaded by a processor and executing the method of the first aspect mentioned above.
  • an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions.
  • the computer instructions When the computer instructions are read and executed by a processor of a computer device, the computer device executes the method of the first aspect involved above.
  • embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium.
  • a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method of the first aspect described above.
  • an embodiment of the present application provides a code stream, which is generated by the method described in the first aspect mentioned above.
  • the prediction method based on template matching includes: determining a first starting search point based on the first vector included in the first vector prediction VP in the vector prediction VP list of the current block; based on the first starting search point, searching for the second vector of the current block in the reference area of the current block using a template formed by reconstructed pixels around the current block; and determining the prediction block of the current block based on the second vector.
  • This is equivalent to the prediction method based on template matching being able to construct a template based on the first starting search point with the help of reconstructed pixels around the current block, searching for the second vector of the current block in the reference area of the current block, and then determining the prediction block of the current block based on the second vector.
  • both the encoding end and the decoding end can use the prediction method based on template matching to determine the prediction block of the current block, that is, the encoding end does not need to transmit vector information to the decoding end, the bit rate occupied by the motion information transmission can be saved, thereby improving the bit rate.
  • searching based on the first starting search point to obtain a second vector and then determining the prediction block of the current block based on the second vector is equivalent to optimizing the first vector, thereby obtaining a higher quality prediction block and improving the prediction effect of the current block.
  • FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application.
  • FIG2 is a schematic block diagram of a video encoder according to an embodiment of the present application.
  • FIG3 is a schematic structural diagram of the relationship between the coding tree unit and the coding unit provided in the present application.
  • FIG4 is a schematic block diagram of a video decoder according to an embodiment of the present application.
  • FIG5 is an example of integer pixels and sub-pixels provided in an embodiment of the present application.
  • FIG6 is an example of a spatial MVP provided in an embodiment of the present application.
  • FIG7 is an example of a three-step search method provided in an embodiment of the present application.
  • FIG8 is an example of a four-step search method provided in an embodiment of the present application.
  • FIG9 is an example of a diamond search method provided in an embodiment of the present application.
  • FIG10 is an example of a hexagonal search method provided in an embodiment of the present application.
  • FIG11 is a schematic flowchart of a prediction method based on template matching provided in an embodiment of the present application.
  • FIG12 is an example of a template provided in an embodiment of the present application.
  • FIG13 is an example of a template of a search point and a template of a current block provided by an embodiment of the present application.
  • FIG14 is an example of a search shape included in the search method provided in an embodiment of the present application.
  • FIG15 is an example of multiple template areas provided in an embodiment of the present application.
  • FIG16 is another example of multiple template areas provided in an embodiment of the present application.
  • FIG17 is a schematic block diagram of a prediction device based on template matching provided in an embodiment of the present application.
  • FIG18 is a schematic block diagram of an electronic device provided in an embodiment of the present application.
  • the solution provided by this application relates to the field of digital compression technology.
  • digital video compression technology mainly compresses huge digital image video data for easy transmission and storage.
  • the solution provided in this application can be applied to the field of digital video coding technology.
  • the field of digital video coding technology includes but is not limited to at least one of the following: image coding and decoding field, video coding and decoding field, hardware video coding and decoding field, dedicated circuit video coding and decoding field, and real-time video coding and decoding field.
  • the solution provided in this application can be combined with the following standards: Audio Video Coding Standard (AVS), the second generation AVS standard (AVS2) or the third generation AVS standard (AVS3).
  • AVS Audio Video Coding Standard
  • AVC H.264/Audio Video Coding
  • HEVC High Efficiency Video Coding
  • VVC Very-Time Video Coding
  • the solution provided in this application can be used for lossy compression of images, and can also be used for lossless compression of images.
  • the lossless compression can be visually lossless compression or mathematically lossless compression.
  • IDR frame In video encoding algorithms, images are organized into sequences. The first image in a sequence is called an IDR frame, and IDR images are all I-frames. IDR frames are also called IDR pictures.
  • IDR frames cause the reference frame list (Decoded Picture Buffer, DPB) to be cleared, while I-frames do not.
  • IDR images are always I-frames, but I-frames are not always IDR frames.
  • a sequence can contain many I-frames, and frames after an I-frame can reference frames between them for motion reference.
  • P-frame A forward predictive coded frame.
  • a P-frame represents the difference between this frame and the previous keyframe (or P-frame).
  • the previously cached image is superimposed on the difference defined by this frame to generate the final image.
  • B-frame A bidirectionally predicted interpolated coded frame.
  • a B-frame is a bidirectional difference frame, which means that a B-frame records the difference between the current frame and the previous and next frames.
  • a B-frame can be used as a reference frame for other B-frames, or it can not be used as a reference frame for other B-frames.
  • Quantization Parameter The rate control algorithm primarily outputs a target bitrate by adjusting the discrete cosine transform (DCT) quantization parameter.
  • DCT discrete cosine transform
  • the QP effectively reflects the compression of spatial detail.
  • a low QP preserves most detail; a high QP loses some detail, lowers the bitrate, and increases image distortion and quality.
  • QP and bitrate are inversely proportional, and this relationship becomes more pronounced as the complexity of the video source increases.
  • Intra-frame prediction The predicted block is a block formed based on the coded reconstructed block and the current block.
  • Inter-frame prediction mainly includes motion estimation (motion search method, motion estimation criteria, sub-pixel interpolation and motion vector estimation) and motion compensation, which is the reference and prediction interpolation compensation at the granularity timing of the Group of Pictures (GOP).
  • motion estimation motion search method, motion estimation criteria, sub-pixel interpolation and motion vector estimation
  • motion compensation which is the reference and prediction interpolation compensation at the granularity timing of the Group of Pictures (GOP).
  • Sum of Absolute Transformed Difference For example, summing the absolute values after a Hadamard transform.
  • Motion Compensation By predicting the motion trajectory of objects in the image and compensating for their displacement, the amount of data between consecutive images can be reduced.
  • Motion Estimation By finding the correlation between adjacent images and calculating the motion vectors between blocks, redundant information in the time dimension is removed, further improving compression efficiency.
  • BD-rate One of the main parameters for evaluating the performance of video encoding algorithms, it indicates the changes in bit rate and Peak Signal-to-Noise Ratio (PSNR) of the video encoded by the new algorithm compared with the original algorithm.
  • PSNR Peak Signal-to-Noise Ratio
  • Rate Distortion Optimization During the encoding process, many modes are available. Some modes offer minimal image distortion but a high bitrate, while others offer high image distortion but a low bitrate. This allows for minimal distortion without exceeding a certain maximum bitrate. In rate-distortion optimization for video encoding, the maximum bitrate is used as a constraint, distortion as the optimization objective, and the Lagrange multiplier method is used to find the optimal encoding parameters.
  • Macroblock The basic unit of encoding. An image must be divided into macroblocks before it can be processed. For example, an image can be divided into macroblocks of a certain size (16 ⁇ 16 in H.264), and the encoding process is performed on a block-by-block basis.
  • Reference image In video coding and decoding, a reconstructed image that serves as a reference for other images and is used by other images to obtain reference data between images during the encoding/decoding process.
  • Motion Vector A two-dimensional vector that describes the position offset that occurs when a coding block moves from its current position to another position in the encoder.
  • Motion Search The process of finding the best matching block for the current coding block in the reference image according to a certain algorithm.
  • Motion Vector Prediction Based on the existing information, the current MV is predicted according to a certain algorithm.
  • a and/or B in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist.
  • a and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
  • the term "at least one" is only a description of the combination relationship of enumerated objects, indicating that one or more items may exist.
  • at least one of the following: A, B, C can mean the following combinations: A exists alone, B exists alone, C exists alone, A and B exist at the same time, A and C exist at the same time, B and C exist at the same time, and A, B, and C exist at the same time.
  • the term “multiple” refers to two or more.
  • the character "/" generally indicates that the related objects before and after are in an "or” relationship.
  • the term “corresponding” can indicate a direct or indirect correspondence between two items, an association between the two items, or a relationship between an indicator and the indicated, a configuration and the configured, and so on.
  • the term “indication” can be a direct indication, an indirect indication, or an association.
  • “A indicates B” can mean that A directly indicates B, for example, B can obtain it through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can obtain it through C; it can also mean that A and B have an association.
  • predefined or “preconfigured” can mean that the corresponding code, table, or other relevant information that can be used for indication is pre-stored in the device, or it can refer to a protocol agreement.
  • Protocol can refer to a standard protocol in this field.
  • the term “when" can be interpreted as “if,” “if,” “when,” “in response to,” and similar descriptions.
  • the phrase “if determined” or “if (stated condition or event) is detected” can be interpreted as “when determined” or “in response to determining” or “when (stated condition or event) is detected” or “in response to detecting (stated condition or event)” and similar descriptions.
  • first, “second”, “third”, “fourth”, “Ath”, “Bth” and the like are used to distinguish different objects rather than to describe a specific order.
  • the terms “including” and “having” and any variations thereof are intended to cover non-exclusive inclusions. Among them, digital video compression technology is mainly used to compress huge digital image video data for easy transmission and storage.
  • FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application.
  • the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120 .
  • the encoding device 110 is used to encode (which can be understood as compressing) the video data to generate a code stream, and transmit the code stream to the decoding device 120.
  • the decoding device 120 decodes the code stream generated by the encoding device 110 to obtain decoded video data.
  • the encoding device 110 can be understood as a device with a video encoding function
  • the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiments of the present application include a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, car computers, etc.
  • the encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via the channel 130 .
  • the encoded video data eg, a code stream
  • Channel 130 may include one or more media and/or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120.
  • Channel 130 may include one or more communication media that enable encoding device 110 to transmit the encoded video data directly to decoding device 120 in real time.
  • Encoding device 110 may modulate the encoded video data according to a communication standard and transmit the modulated video data to decoding device 120.
  • Communication media may include wireless communication media, such as radio frequency spectrum.
  • Communication media may also include wired communication media, such as one or more physical transmission lines.
  • Channel 130 may include a storage medium that can store the video data encoded by encoding device 110. Storage media include various locally accessible data storage media, such as optical disks, DVDs, and flash memory.
  • decoding device 120 may retrieve the encoded video data from the storage medium.
  • Channel 130 may also include a storage server that can store the video data encoded by encoding device 110.
  • decoding device 120 may download the stored encoded video data from the storage server.
  • the storage server may store the encoded video data and transmit the encoded video data to the decoding device 120 , such as a web server (eg, for a website), a File Transfer Protocol (FTP) server, etc.
  • a web server eg, for a website
  • FTP File Transfer Protocol
  • the encoding device 110 includes a video encoder 112 and an output interface 113.
  • the output interface 113 may include a modulator/demodulator (modem) and/or a transmitter.
  • the video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113.
  • the encoded video data may also be stored on a storage medium or storage server for subsequent reading by the decoding device 120.
  • encoding device 110 may also include a video source 111.
  • Video source 111 may include at least one of a video capture device (e.g., a video camera), a video archive, a video input interface, and a computer graphics system.
  • the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data.
  • Video encoder 112 encodes the video data from video source 111 to generate a bitstream.
  • the video data may include one or more pictures or a sequence of pictures.
  • the bitstream contains encoding information for the picture or picture sequence in the form of a bitstream.
  • the encoding information may include encoded picture data and associated data.
  • Associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures.
  • SPS sequence parameter set
  • PPS picture parameter set
  • An SPS may contain parameters applicable to one or more sequences.
  • a PPS may contain parameters applicable to one or more pictures.
  • a syntax structure is a set of zero or more syntax elements arranged in a specified order within a bitstream.
  • the decoding device 120 includes an input interface 121 and a video decoder 122.
  • the input interface 121 may include a receiver and/or a modem.
  • the input interface 121 may receive encoded video data via a channel 130.
  • the video decoder 122 is configured to decode the encoded video data to obtain decoded video data.
  • the decoding device 120 may also include a display device 123.
  • the video decoder 122 transmits the decoded video data to the display device 123.
  • the display device 123 displays the decoded video data.
  • the display device 123 may be integrated with the decoding device 120 or external to the decoding device 120.
  • the display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.
  • LCD liquid crystal display
  • OLED organic light emitting diode
  • Figure 1 is only an example of the present application and should not be understood as a display of the present application. That is to say, the technical solution of the embodiment of the present application is not limited to the system framework shown in Figure 1.
  • the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.
  • FIG2 is a schematic block diagram of a video encoder 200 according to an embodiment of the present application.
  • the video encoder 200 can be applied to image data in luminance and chrominance (YCbCr, YUV) format.
  • the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) used to describe color and saturation.
  • 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr)
  • 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr)
  • 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr).
  • RGB red-green-blue
  • the video encoder 200 After reading the video stream, the video encoder 200 divides each frame into a number of coding tree units (CTUs).
  • CTUs may be referred to as a tree block, largest coding unit (LCU), or coding tree block (CTB).
  • LCU largest coding unit
  • CTB coding tree block
  • a CTU can be sized, for example, 128 ⁇ 128, 64 ⁇ 64, or 32 ⁇ 32.
  • FIG3 is a schematic structural diagram of the relationship between the coding tree unit and the coding unit provided in the present application.
  • a CTU can be further divided into several coding units (CUs) for encoding.
  • a CU can be either rectangular or square.
  • a CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing.
  • PUs prediction units
  • TUs transform units
  • a CTU is divided into CUs using a tree (e.g., a quadtree)
  • a CU is divided into TUs and PUs using a tree (e.g., a quadtree).
  • the video encoder and video decoder can support various PU sizes.
  • the video encoder and video decoder may support PU sizes of 2N ⁇ 2N or N ⁇ N for intra prediction, and support symmetric PUs of 2N ⁇ 2N, 2N ⁇ N, N ⁇ 2N, N ⁇ N, or similar sizes for inter prediction.
  • the video encoder and video decoder may also support asymmetric PUs of 2N ⁇ nU, 2N ⁇ nD, nL ⁇ 2N, and nR ⁇ 2N for inter prediction.
  • the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform/quantization unit 230, an inverse transform/quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components.
  • a current block may be referred to as a current coding unit (CU) or a current prediction unit (PU).
  • a prediction block may also be referred to as a predicted image block or an image prediction block, and a reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.
  • the prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Because there is a strong correlation between adjacent pixels in a video image, intra-frame prediction is used in video coding and decoding to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent images in a video, inter-frame prediction is used to eliminate temporal redundancy between adjacent images, thereby improving coding efficiency.
  • the inter-frame prediction unit 211 is capable of inter-frame prediction, which includes motion estimation and motion compensation. It can reference image information from different frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and then generates a prediction block based on the reference block to eliminate temporal redundancy.
  • the reference frame can be a P-frame and/or a B-frame. P-frames refer to forward-predicted frames, and B-frames refer to bidirectionally predicted frames. After using motion information to find a reference block, inter-frame prediction generates a prediction block based on the reference block.
  • Motion information includes the frame list to which the reference frame belongs, the frame index, and a motion vector. The motion vector can be integer-pixel or fractional-pixel.
  • interpolation filtering is used in the reference frame to generate the required fractional-pixel block.
  • the reference block is the integer-pixel or fractional-pixel block found based on the motion vector. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate a prediction block. "Reprocessing a reference block to generate a prediction block” can also be understood as using the reference block as the prediction block and then processing the prediction block to generate a new prediction block.
  • the intra-frame prediction unit 212 only refers to the information of the same frame image to predict the pixel information in the current code image block to eliminate spatial redundancy.
  • the reference frame used for intra-frame prediction can be an I frame.
  • Intra-frame prediction uses multiple prediction modes. Both angular and non-angular prediction modes can be used to predict the image block to be coded, yielding a prediction block. Based on the rate-distortion information calculated from the prediction block and the image block to be coded, the optimal prediction mode for the image block to be coded is selected and written into the bitstream for transmission to the decoder. The decoder parses the prediction mode, predicts the predicted block for the target decoding block, and overlays it with the time-domain residual block obtained from the bitstream to reconstruct the block.
  • the H.264/AVC standard has eight angular prediction modes and one non-angle prediction mode, while H.265/HEVC has expanded this to 33 angular prediction modes and two non-angle prediction modes.
  • HEVC uses planar, direct current (DC), and 33 angular modes, for a total of 35 intra-frame prediction modes.
  • VVC uses planar, DC, and 65 angular modes, for a total of 67 prediction modes, including traditional and non-traditional prediction modes.
  • Non-traditional prediction modes can include matrix-weighted intra-frame prediction (MIP) mode.
  • Traditional prediction modes include planar mode (mode 0), DC mode (mode 1), and angular prediction modes (modes 2 through 66).
  • intra-frame prediction mode is only an example of this application and should not limit this application.
  • the residual unit 220 may generate a residual block for the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.
  • the transform/quantization unit 230 may quantize the transform coefficients.
  • the transform/quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU.
  • QP quantization parameter
  • the video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.
  • the inverse transform/quantization unit 240 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.
  • Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 can reconstruct the pixel blocks of the CU.
  • the loop filter unit 260 processes the inverse-transformed and inverse-quantized pixels to compensate for distortion and provide a better reference for subsequent pixel encoding. For example, it can perform deblocking filtering to reduce the blocking artifacts of pixel blocks associated with the CU.
  • the loop filter unit 260 includes a deblocking filter (DBF) unit and a sample adaptive offset/adaptive loop filter (SAO/ALF) unit.
  • DBF deblocking filter
  • SAO/ALF sample adaptive offset/adaptive loop filter
  • the decoded image buffer 270 can store reconstructed pixel blocks.
  • the inter-frame prediction unit 211 can use the reference image containing the reconstructed pixel blocks in the decoded image buffer 270 to perform inter-frame prediction on PUs in other images.
  • the intra-frame prediction unit 212 can use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra-frame prediction on other PUs in the same image as the CU.
  • the entropy coding unit 280 may receive the quantized transform coefficients from the transform/quantization unit 230.
  • the entropy coding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy-coded data.
  • FIG4 is a schematic block diagram of a video decoder according to an embodiment of the present application.
  • the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization/transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.
  • the entropy decoding unit 310 can parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 can parse the syntax elements in the bitstream that have been entropy-encoded.
  • the prediction unit 320, the inverse quantization/transform unit 330, the reconstruction unit 340, and the loop filter unit 350 can decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.
  • the prediction unit 320 includes an intra prediction unit 322 and an inter prediction unit 321.
  • the intra prediction unit 322 can perform intra prediction to generate a prediction block for the PU.
  • the intra prediction unit 322 can use an intra prediction mode to generate a prediction block for the PU based on pixel blocks of spatially neighboring PUs.
  • the intra prediction unit 322 can also determine the intra prediction mode for the PU based on one or more syntax elements parsed from the codestream.
  • the inter prediction unit 321 can construct a first reference image list (List 0) and a second reference image list (List 1) based on the syntax elements parsed from the codestream.
  • the entropy decoding unit 310 can parse the motion information of the PU.
  • the inter prediction unit 321 can determine one or more reference blocks for the PU based on the motion information of the PU.
  • the inter prediction unit 321 can generate a prediction block for the PU based on the one or more reference blocks of the PU.
  • the inverse quantization/transform unit 330 may inversely quantize (i.e., dequantize) the transform coefficients associated with the TU.
  • the inverse quantization/transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization. After inverse quantizing the transform coefficients, the inverse quantization/transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients to generate a residual block associated with the TU.
  • the reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.
  • the loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.
  • the video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360.
  • the video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.
  • a frame of an image is divided into image blocks.
  • the prediction unit 210 uses intra-frame prediction or inter-frame prediction to predict the prediction block of the current block (i.e., the block to be encoded).
  • the residual unit 220 can calculate a residual block based on the predicted block and the original block of the current block (i.e., the block to be encoded), which is the difference between the predicted block and the original block.
  • This residual block can also be referred to as residual information.
  • This residual block is transformed and quantized by the transform/quantization unit 230, thereby removing information that is insensitive to the human eye and eliminating visual redundancy.
  • the residual block before transformation and quantization by the transform/quantization unit 230 can be referred to as a time-domain residual block
  • the time-domain residual block after transformation and quantization by the transform/quantization unit 230 can be referred to as a frequency residual block or a frequency-domain residual block.
  • the entropy coding unit 280 receives the quantized change coefficients output by the change quantization unit 230 and can perform entropy coding on the quantized change coefficients to output a bitstream.
  • the entropy coding unit 280 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.
  • the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block (i.e., the block to be decoded).
  • the prediction unit 320 uses intra-frame prediction or inter-frame prediction based on the prediction information to predict the prediction block of the current block (i.e., the block to be decoded).
  • the inverse quantization/transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block.
  • the reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block.
  • the reconstructed blocks constitute a reconstructed image
  • the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. It is worth noting that the encoding end also needs to use operations similar to those of the decoder to obtain a decoded image.
  • the decoded image can also be called a reconstructed image, and the reconstructed image can be a subsequent image, which serves as a reference image for inter-frame prediction.
  • the block division information determined by the encoder as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary.
  • the decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are the same.
  • images can be divided into slices, etc., and slices in the same image can be processed in parallel, that is, there is no data dependency between them.
  • frame can be understood as an image or slice, etc.
  • Video signals can be divided into two types based on how they are acquired: those captured by cameras and those generated by computers. Due to differences in statistical properties, the corresponding compression encoding methods may also differ.
  • Video coding technology taking the international video coding standards HEVC and VVC, and China's national video coding standard AVS as examples, adopts a hybrid coding framework to perform the following operations and processing on the input raw video signal:
  • Block partition structure The input image is divided into several non-overlapping processing units based on their size. Each processing unit performs similar compression operations. This processing unit is called a CTU or LCU. Below the CTU, further subdivision can be performed to obtain one or more basic coding units, called CUs. Each CU is the most basic element in the encoding process. The following describes the various possible encoding methods for each CU.
  • Predictive Coding This includes intra-frame prediction and inter-frame prediction.
  • the original video signal is predicted by the selected reconstructed video signal to obtain a residual video signal.
  • the encoder selects the most suitable predictive coding mode for the current CU from among many possible predictive coding modes and informs the decoder.
  • Intra-frame prediction The predicted signal comes from an already encoded and reconstructed area within the same image.
  • Inter-frame prediction The predicted signal comes from an already encoded image that is different from the current image (called a reference image).
  • Transform Coding and Quantization (Transform & Quantization): The residual video signal undergoes transformation operations such as DFT and DCT to convert the signal into the transform domain, which is called the transform coefficient. The signal in the transform domain is further subjected to lossy quantization operations, which loses a certain amount of information, making the quantized signal conducive to compression expression. In some video coding standards, there may be more than one transform method to choose from. Therefore, the encoder also needs to select one of the transforms for the current coding CU and inform the decoder. The degree of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a larger value range will be quantized into the same output, which usually results in greater distortion and a lower bit rate. Conversely, a smaller QP value means that coefficients with a smaller value range will be quantized into the same output, which usually results in less distortion and a corresponding higher bit rate.
  • QP quantization parameter
  • Entropy Coding or Statistical Coding The quantized transform domain signal will be statistically compressed and encoded according to the frequency of occurrence of each value, and finally a binary (0 or 1) compressed code stream will be output. At the same time, the encoding generates other information, such as the selected mode, motion vector, etc., which also needs to be entropy coded to reduce the bit rate.
  • Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).
  • Loop Filtering After the encoded image undergoes inverse quantization, inverse transformation, and prediction compensation (the reverse operations of 2 to 4 above), a reconstructed decoded image can be obtained. Compared with the original image, the reconstructed image has some information that is different from the original image due to the influence of quantization, resulting in distortion. Filtering operations on the reconstructed image, such as deblocking filtering, SAO or ALF filters, can effectively reduce the degree of distortion caused by quantization. Since these filtered reconstructed images will be used as a reference for subsequent encoded images and used to predict future signals, the above filtering operation is also called loop filtering, and the filtering operation within the encoding loop.
  • deblocking filtering SAO or ALF filters
  • the encoding process described above shows that, at the decoding end, after receiving the compressed bitstream for each CU, the decoder first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transformation to produce a residual signal. Separately, based on the known coding mode information, the prediction signal corresponding to that CU is obtained. Adding these two together yields the reconstructed signal. Finally, the reconstructed value of the decoded image undergoes loop filtering to produce the final output signal.
  • the predicted pixels for inter-frame prediction or intra-frame block copy mode are obtained by searching in the reconstructed reference frame or the reconstructed area of the current frame, or by searching in its sub-pixel interpolated image.
  • the chrominance component it is generally assumed that it has a similar motion vector or block vector as the luminance component, so the chrominance motion vector or block vector is simply derived based on the luminance motion vector or block vector. Since this matching search process not only considers the image matching degree, but also the coding cost of the motion vector or block vector, the matching degree alone may not be optimal, which means that there is still room for improvement in prediction accuracy.
  • the reference frame can be divided into forward reference frames and backward reference frames according to the relationship between the reference frame and the current frame in the time domain.
  • Motion information generally includes a reference frame index and a motion vector (MV). If a motion information contains 1 motion information, it is called unidirectional motion information; if it contains 2 motion information, it is called bidirectional motion information. Note that bidirectional motion information does not mean that there is forward motion information and backward motion information, it may also be both forward.
  • the motion information of the intra-frame block copy mode will not have a reference frame index.
  • FIG5 is an example of integer pixels and sub-pixels provided in an embodiment of the present application.
  • the black squares represent integer pixels (the resolution of all integer pixels in a frame is equal to the video resolution).
  • the movement of an object or the change of its texture is not necessarily an integer pixel, so the related motion vector may not be an integer pixel and may point to the 1/4 pixel position of the gray circle in the figure.
  • Temporal Motion Vector Prediction This means that the motion vector is derived from a reference frame.
  • the MV of the coding block corresponding to the current block (intra-block or extra-block sub-block) in the temporal reference frame can be selected (directly or after a simple offset) as the motion vector prediction for the current block.
  • Spatial Motion Vector Prediction This means the motion vector comes from the current frame (encoded/decoded area).
  • the MV of the coded block in the coded area surrounding the current block can be selected (directly or after a simple offset) as the motion vector prediction for the current block.
  • AVS3 as shown in Figure 6, a bidirectional, backward, and forward motion information is searched from the sub-blocks F, G, C, A, B, and D outside the block and added to the motion vector prediction list.
  • HMVP History based Motion Vector Prediction
  • HMVP technology copies 8 motion information candidates from the previous coding block to the FIFO, and the FIFO is constantly updated in a first-in, first-out manner. If the motion information candidate in the FIFO is the same as the motion information just encoded, the duplicate candidate will be removed first, and the motion information of the current coding unit will be added to the end of the FIFO. If the motion information of the current coding unit is different from the motion information of any candidate in the FIFO, the first candidate in the FIFO will be removed, and the latest motion information will be added to the end of the FIFO to ensure that the 8 latest motion candidates are always retained in the FIFO.
  • Three-step search method Use the search graph shown in Figure 7 to search, and each search step is halved. A total of three iterative searches are performed. In each iterative search, the search point with the smallest SAD is selected as the starting point for the next iterative search.
  • Four-step search method Use the search graph shown in Figure 8 (a) to search. First, the search image is 5x5, and the iterative search is performed until the center is optimal. Then, the search image is scaled to the search graph shown in Figure 8 (b), and another search is performed to obtain the optimal search point.
  • Diamond search method Use the search graph shown in Figure 9 (a) to search, iterate the search until the center is optimal, then use the search graph shown in Figure 9 (b) to continue searching, and then search again to obtain the optimal search point.
  • Hexagonal search method Use the search graph shown in (a) in Figure 10 to search, iterate the search until the center is optimal, then use the search graph shown in (b) in Figure 10 to continue searching once, and then use the search graph shown in (c) in Figure 10 to continue searching once to obtain the best search point.
  • data collection and processing in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and personal information subjects.
  • data collection can be achieved in the form of collecting logs, for example, by obtaining the logs of the target memory or target processor, the relevant data can be obtained.
  • the collection granularity can be in units of days or weeks, etc., and this application does not make specific restrictions on this.
  • the prediction method can be applied to products with relevant video codecs or video compression.
  • the prediction method provided by an embodiment of the present application can be executed by an encoding end, such as the encoding device 110 or video encoder 112 shown in Figure 1, and then the video encoder 200 shown in Figure 2.
  • the prediction method provided by an embodiment of the present application can be executed by a decoding end, such as the decoding device 120 or video decoder 122 shown in Figure 1, and then the video decoder 300 shown in Figure 4.
  • the prediction method is described below using a prediction device as an example.
  • FIG11 is a schematic flowchart of a prediction method 400 based on template matching according to an embodiment of the present application.
  • the prediction method 400 may include at least part of the following:
  • the prediction apparatus determines a first starting search point based on a first vector included in a first vector prediction VP in a vector prediction VP list of a current block.
  • the VP list includes, but is not limited to, a motion vector prediction (MVP) list or a block vector prediction (BVP) list.
  • MVP motion vector prediction
  • BVP block vector prediction
  • the VP list includes the MVP list
  • the first VP is the first MVP and the first vector is the MV.
  • the VP list includes the BVP list
  • the first VP is the BVP and the first vector is the BV.
  • the prediction apparatus searches for a second vector of the current block in a reference area of the current block based on the first starting search point and using a template formed by reconstructed pixels around the current block.
  • the template of the current block may be composed of reconstructed pixels surrounding the current block, and the reconstructed pixels surrounding the current block include reconstructed pixels at at least one of the following positions around the current block: left, top, upper left, lower left, and upper right.
  • FIG12 shows eight examples of templates composed of reconstructed pixels surrounding the current block.
  • the present application is not limited to the template examples shown in FIG12.
  • the reference region of the current block is related to the prediction mode of the current block.
  • the prediction mode of the current block may be inter-frame prediction or intra-frame block copy mode.
  • the prediction mode of the current block is inter-frame prediction mode
  • the reference region is located within the reference frame of the current block; when the prediction mode of the current block is intra-frame block copy mode, the reference region is located within the reconstructed region of the current image to which the current block belongs.
  • the reference region may also be referred to as a search region.
  • the prediction device searches for the second vector of the current block within the reference area of the current block based on the first starting search point and using the cost of the template of the current block relative to the template of each candidate search point. For example, the prediction device searches for the second vector of the current block within at least one candidate search point within the reference area of the current block based on the first starting search point and using the cost of the template formed by reconstructed pixels surrounding the current block relative to the template of each candidate search point.
  • the template of the candidate search point involved in this application can be understood as a template matched based on the candidate search point in the template matching process or a template searched based on the candidate search point, and this application does not limit the specific description thereof.
  • the prediction device determines a prediction block of the current block based on the second vector.
  • the prediction device determines the prediction block of the current block based on the first prediction block pointed to by the second vector.
  • a first starting search point is determined based on the first vector included in the first vector prediction VP in the vector prediction VP list of the current block; based on the first starting search point, a second vector of the current block is searched within the reference area of the current block using a template formed by reconstructed pixels around the current block; and based on the second vector, the predicted block of the current block is determined.
  • This is equivalent to the prediction method based on template matching being able to construct a template based on the first starting search point with the help of reconstructed pixels around the current block, search for the second vector of the current block within the reference area of the current block, and then determine the predicted block of the current block based on the second vector.
  • both the encoding end and the decoding end can use the prediction method based on template matching to determine the predicted block of the current block, that is, the encoding end does not need to transmit vector information to the decoding end, the bit rate occupied by the motion information transmission can be saved, thereby improving the bit rate.
  • searching based on the first starting search point to obtain a second vector and then determining the prediction block of the current block based on the second vector is equivalent to optimizing the first vector, thereby obtaining a higher quality prediction block and improving the prediction effect of the current block.
  • the first VP is indicated by a first identifier, which is carried by a bitstream or determined by derivation at a decoding end; or, the first VP is a VP with the smallest template cost in the VP list.
  • the identifier (such as the first identifier or other identifier) is determined by deduction at the decoding end, which is intended to illustrate that both the encoding end and the decoding end can determine the identifier by deduction, or in other words, the encoding end does not need to write the identifier into the bitstream, and the decoding end does not need to obtain the identifier by decoding the bitstream.
  • the identifier is determined by deduction at the decoding end, which can also be equivalently replaced by: the identifier is determined by implicit indication, or the identifier is determined by implicit deduction.
  • the identifier is carried by the bitstream, which is intended to illustrate that the decoding end writes the identifier into the bitstream, and the decoding end obtains the identifier by decoding the bitstream.
  • the first VP may be the VP with the lowest template cost in the VP list, or may be another VP in the VP list, and this application does not specifically limit this.
  • the first VP may be a VP determined in the VP list based on the size of the current block or other information.
  • the search for the second vector in the present application can also be equivalently replaced by a search for the first prediction block pointed to by the second vector, or equivalently replaced by a search for the template of the first prediction block pointed to by the second vector, and the present application does not make specific limitations on this.
  • the prediction device can determine a first starting search point based on the first vector included in the first VP in the VP list of the current block; based on the first starting search point, use a template formed by reconstructed pixels around the current block to search for the first prediction block in the reference area of the current block; and determine the prediction block of the current block based on the first prediction block vector.
  • the codeword length of the first identifier is positively correlated with the template cost of the first VP.
  • the prediction device when it functions as an encoder, it may evaluate the template costs of the VPs in the VP list and then sort the VPs in the VP list in ascending order of template cost, such that the VP with the smallest template cost corresponds to the shortest codeword length. For example, assuming there are four VPs in the VP list, then in ascending order of template cost, when the first identifier indicates the four VPs in the VP list, their corresponding codewords may be 1, 01, 001, and 000, respectively.
  • the codeword length of the first identifier may have no relationship with the template cost of the first VP.
  • the codeword length of the first identifier is determined according to the number of VP lists in the VP list, or is a fixed length.
  • the method 400 before S410, the method 400 further includes:
  • An initial list is constructed based on at least one of adjacent spatial domain VPs, time domain VPs, non-adjacent spatial domain VPs, historical VPs, average VPs, and zero vector VPs; and at least one of deduplication, vector adjustment, template cost evaluation, and template cost-based sorting is performed on the initial VPs in the initial list to obtain the VP list.
  • neighboring spatial VP refers to predicting the vector of the current block based on the vectors of decoded blocks spatially adjacent to the current block. Due to continuity in video sequences, the vectors of neighboring blocks are often similar to the vector of the current block.
  • Temporal VP refers to predicting the vector of the current block using the vectors of correspondingly located blocks in the decoded reference image. This helps capture motion coherence across multiple frames.
  • Non-neighboring spatial VP refers to a method similar to neighboring spatial VP, but considers more distant or less direct neighboring blocks. These blocks may be located in the same row or column as the current block or determined based on a search pattern.
  • Historical VP refers to predicting the vector of the current block using the vectors of decoded blocks (possibly in different frames or coding units). This method can capture motion patterns or periodic motion over a longer time span. Average VP refers to averaging the vectors of multiple decoded blocks (possibly adjacent or non-adjacent). This method helps smooth out changes in motion vectors and reduce prediction errors. Zero vector VP refers to using a zero vector as the predicted vector for blocks with little motion or static blocks. This helps reduce the number of bits required to encode small moving or stationary blocks.
  • Neighboring spatial MVP predicts the motion vector of the current block based on the motion vectors of spatially neighboring decoded blocks. Due to continuity in video sequences, the motion vectors of neighboring blocks are often similar to the motion vector of the current block. Temporal MVP predicts the motion vector of the current block using the motion vector of the corresponding block in the decoded reference image. This helps capture motion coherence across multiple frames. Non-neighboring spatial MVP is similar to neighboring spatial MVP but considers more distant or less direct neighboring blocks. These blocks may be located in the same row or column as the current block or determined based on a search pattern.
  • Historical MVP predicts the motion vector of the current block using the motion vectors of decoded blocks (possibly in different frames or coding units). This method can capture motion patterns or periodic motion over a long time span. Average MVP calculates the predicted motion vector by averaging the motion vectors of multiple decoded blocks (possibly adjacent or non-adjacent). This method helps smooth motion vector variations and reduce prediction errors. Zero-vector MVP means that for blocks with little motion or stillness, a zero vector is used as the predicted motion vector. This helps reduce the number of bits required to encode blocks with little motion or stillness.
  • construction process of the initial list in this application may refer to the construction process of the MVP list or the construction process of the BVP list.
  • the initial list can be constructed using the MVP list construction process of the inter-frame prediction mode merge/skip/direct mode/advanced motion vector prediction (AMVP) mode.
  • the initial list can be constructed based on the coded blocks around the current block (i.e., spatial MVP), the coded blocks at the temporal associated position (i.e., temporal MVP), the history MVP (History-based Motion Vector Prediction, HMVP), and the zero vector MVP.
  • the temporal MV can be selected to construct two unidirectional motion information (pointing to the forward and backward reference frame lists, respectively) to form an MVP to construct the initial list.
  • a MVP list of length 4 is constructed, which is: a time domain MVP, a bidirectional MVP constructed by adjacent spatial domain MVPs, a backward MVP and a forward MVP.
  • the prediction device may perform deduplication processing on the initial VPs in the initial list, that is, delete repeated initial VPs in the initial list.
  • the prediction device may adjust the vector of the initial VP in the initial list.
  • the prediction device may adjust the position pointed to by the initial VP in the initial list.
  • the prediction device may adjust at least one dimension of the vector of the initial VP in the initial list.
  • the prediction device may perform template cost evaluation on the initial VPs in the initial list.
  • the prediction device may sort the initial VPs in the initial list based on the template cost. For example, the prediction device may evaluate the template cost of the initial VPs in the initial list and then sort the initial VPs in the initial list based on the template cost.
  • the order of deduplication, vector adjustment, template cost evaluation, and template cost-based sorting can be flexibly adjusted.
  • the order of execution can be: deduplication, template cost evaluation, vector adjustment; it can also be: template cost evaluation, vector adjustment, deduplication, etc.; or it can also be: deduplication, vector adjustment, template cost evaluation.
  • the S410 includes:
  • At least one dimension of the first vector is adjusted to obtain an adjusted vector of the first vector, and a position pointed to by the adjusted vector is determined as the first starting search point.
  • the at least one dimension corresponds to at least one numerical value of the first vector.
  • the prediction device may adjust the at least one numerical value of the first vector to obtain an adjusted vector for the first vector, and determine the position pointed to by the adjusted vector as the first starting search point.
  • the at least one dimension may be determined by protocol agreement, carried in the bitstream, or derived by the decoder.
  • adjusting at least one dimension of the first vector to obtain a first adjusted vector of the first vector includes at least one of the following:
  • At least one dimension of the first vector is offset to obtain the adjustment vector.
  • the VD of the first vector is the motion vector difference (MVD).
  • the VD of the first vector is the block vector difference (BVD).
  • the VD of the first vector is MVD, that is, the prediction device can perform template matching based on (MV+MVD) to obtain a better MV (i.e., the second vector).
  • the MVD is obtained by parsing the code stream.
  • the VD of the first vector is BVD, that is, the prediction device can perform template matching based on (BV+BVD) to obtain a better BV (i.e., the second vector).
  • the BVD is obtained by parsing the code stream.
  • the prediction device may determine the number of shift bits based on the first vector, perform a right shift calculation on at least one dimension of the first vector based on the number of shift bits, and perform a left shift calculation on the result of the right shift based on the number of shift bits to obtain the adjustment vector.
  • the prediction device may directly clear the sub-pixel values of the first vector to zero, so that the first vector is an integer pixel value.
  • the prediction device may determine the number of bits corresponding to the precision of the first vector and the precision of the first number as the number of shift bits. For example, taking the first vector as (MV_x, MV_y) as an example, if the precision of the first vector is 1/4 pixel, the prediction device may perform a shift operation on each dimension of the first vector in the following manner to obtain the adjustment vector:
  • the prediction device may adjust at least one dimension of the first vector to point to the integer pixel position closest to the position pointed by the first vector, thereby obtaining the adjusted vector.
  • the position pointed by the first vector may also be referred to as the original pointing position, i.e., the prediction device may adjust at least one dimension of the first vector to point to the integer pixel closest to the original pointing position.
  • the prediction device may adjust at least one dimension of the first vector to point to another sub-pixel position closest to the position pointed by the first vector, thereby obtaining the adjusted vector, and this application does not specifically limit this.
  • the prediction device may offset at least one dimension of the first vector based on a preset offset to obtain the adjusted vector. For example, taking the first vector (MV_x, MV_y) as an example, the prediction device may offset MV_x and MV_y by 1/4, 1/2, or 3/4 pixels, respectively, to obtain the adjusted vector.
  • the prediction device may determine a shift bit number based on the first vector, perform a right shift calculation on the first dimension of the first vector based on the shift bit number, and perform a left shift calculation on the result of the right shift based on the shift bit number. Furthermore, the prediction device may offset the second dimension of the first vector based on a preset offset to obtain the adjustment vector. For example, where the first dimension is a fractional pixel value and the second dimension is an integer pixel value, the prediction device may determine a shift bit number based on the first vector, perform a right shift calculation on the first dimension of the first vector based on the shift bit number, and perform a left shift calculation on the result of the right shift based on the shift bit number.
  • the prediction device may offset the second dimension of the first vector based on a preset offset to obtain the adjustment vector. For example, for a 1/4 pixel precision first vector (MV_x, MV_y), if one dimension is an integer pixel value, a +2 or -2 adjustment is performed to a 1/2 pixel position; if the other dimension is a fractional pixel value, a right shift of 2 bits is performed followed by a left shift of 2 bits to obtain the adjustment vector.
  • MV_x, MV_y if one dimension is an integer pixel value, a +2 or -2 adjustment is performed to a 1/2 pixel position; if the other dimension is a fractional pixel value, a right shift of 2 bits is performed followed by a left shift of 2 bits to obtain the adjustment vector.
  • the prediction device may adjust at least one dimension of the first vector to point to an integer pixel position, and then adjust it to point to a fractional pixel position. For example, the prediction device may determine a shift bit number based on the first vector, perform a right shift calculation on at least one dimension of the first vector based on the shift bit number, and perform a left shift calculation on the right-shifted calculation result based on the shift bit number, and then offset at least one dimension of the left-shifted vector based on a preset offset to obtain the adjusted vector.
  • the prediction device may adjust at least one dimension of the first vector to point to the integer pixel position closest to the position pointed to by the first vector, and then offset at least one dimension of the vector pointing to the integer pixel position closest to the position pointed to by the first vector based on a preset offset to obtain the adjusted vector.
  • adjusting the dimension of the first vector can be understood as adjusting the value of the first vector in the dimension.
  • the prediction device can also offset at least one dimension of the first vector based on a preset rule to obtain the adjusted vector, which is not specifically limited in this application.
  • the S420 includes:
  • the first vector is adjusted to point to a first optimal search point with the minimum template cost among the at least one candidate search point, to obtain the second vector.
  • the prediction device may directly determine the first vector as the second vector; when the first candidate search point is not the first optimal search point, the prediction device adjusts the first vector to point to the first optimal search point to obtain the second vector.
  • searching for at least one candidate search point based on the first starting search point includes at least one of the following:
  • the precondition includes at least one of the following:
  • the candidate search point with the minimum template cost obtained in the current iteration process is used as the starting search point used in the current iteration process;
  • the number of iterations is greater than or equal to the maximum number of iterations
  • the number of points searched is greater than or equal to the maximum number of search points.
  • the number of points searched is greater than or equal to the maximum number of search points, which can be understood as: the number of points searched is greater than or equal to the maximum number of search points, for example, the number of points searched and deduplicated is greater than or equal to the maximum number of search points, and for another example, the number of points searched and including duplicates is greater than or equal to the maximum number of search points.
  • the prediction device may obtain the at least one candidate search point by traversing all candidate search points in the reference area.
  • the prediction device obtains the at least one candidate search point search method by traversing the candidate search points. This is equivalent to the prediction device traversing all possible candidate search points in the reference area to find the first optimal search point with the minimum template cost.
  • the prediction device may iteratively search for a candidate search point with the lowest template cost within the reference area based on the first starting search point and the first search shape, obtaining a candidate search point obtained in the current iteration, and determining the candidate search point with the lowest template cost obtained in the current iteration as the first optimal search point until a preset condition is satisfied. It is noteworthy that when the prediction device performs an iterative search, the at least one candidate search point includes candidate search points obtained in each iteration. The candidate search points obtained in different iterations may or may not overlap, and this is not specifically limited in this application.
  • the prediction device after the prediction device obtains the at least one candidate search point by traversing the search candidate points, it can perform a template cost evaluation on the at least one candidate search point and adjust the first vector to point to the first optimal search point with the smallest template cost among the at least one candidate search point to obtain the second vector.
  • the prediction device iteratively searches for the candidate search point with the smallest template cost to obtain the candidate search point obtained in the current iteration process, it can deduplicate the candidate search points obtained in the current iteration process, for example, removing the candidate search points that have been excluded before, and then perform template cost evaluation on the deduplicated candidate search points, and adjust the first vector to point to the first best search point with the smallest template cost among the deduplicated candidate search points to obtain the second vector.
  • each time a candidate search point template is searched the difference between the template and the current block template is calculated.
  • the difference is evaluated using the sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), and mean squared difference (MSD).
  • SAD sum of absolute differences
  • SSD sum of squared differences
  • MAD mean absolute difference
  • MSD mean squared difference
  • the cutoff condition for the iterative search performed by the prediction device includes at least one of the following:
  • the candidate search point with the minimum template cost obtained in the current iteration process is used as the starting search point used in the current iteration process;
  • the number of iterations is greater than or equal to the maximum number of iterations
  • the number of points searched is greater than or equal to the maximum number of search points.
  • cut-off conditions can be used individually or in combination.
  • the prediction device can iteratively search the candidate search point with the smallest template cost based on the first starting search point and the first search shape used by the current block to obtain the candidate search point obtained in the current iteration process, until the candidate search point with the smallest template cost obtained in the current iteration process is the starting search point used in the current iteration process, and then determine the candidate search point with the smallest template cost obtained in the current iteration process as the first optimal search point.
  • the prediction device may iteratively search the candidate search point with the smallest template cost based on the first starting search point and the first search shape used by the current block to obtain the candidate search point obtained in the current iteration process, and determine the candidate search point with the smallest template cost obtained in the current iteration process as the first optimal search point until any one of the following conditions is met.
  • the candidate search point with the minimum template cost obtained in the current iteration process is used as the starting search point used in the current iteration process;
  • the number of iterations is greater than or equal to the maximum number of iterations
  • the number of points searched is greater than or equal to the maximum number of search points.
  • an iterative search is performed on a candidate search point with the minimum template cost to obtain the candidate search point obtained in the current iteration, including:
  • an iterative search is performed on the candidate search point with the smallest template cost to obtain the candidate search point obtained in the current iterative process; wherein the search method includes at least one search shape and the number of iterations corresponding to each search shape of the at least one search shape, and the at least one search shape includes the first search shape.
  • the prediction device iteratively searches for candidate search points with the lowest template cost, effectively striking a balance between search speed and finding the optimal template prediction block.
  • Template matching algorithms are similar to motion search methods used on the encoder side. Common methods include three-step search, four-step search, diamond search, hexagonal search, and asymmetric cross-shaped multi-level hexagonal grid search.
  • FIG14 is an example of a search shape included in the search method provided in an embodiment of the present application.
  • the search shape included in the search method can be any of the shapes in (a) to (g).
  • the five-pointed star in each shape is the starting search point used in an iterative process, and the five-pointed star and all squares are candidate search points to be searched in an iterative process.
  • the candidate search point with the minimum template cost is obtained, which is called an iterative process.
  • the current iterative process in this application can be understood as the current (or current round) iterative process or the currently executed iterative process.
  • the candidate search point with the smallest template may be searched through multiple iterative processes.
  • the prediction device determines the candidate search point with the minimum template cost obtained in the current iteration as the first optimal search point if a preset condition is met. If the preset condition is not met, the iteration process is re-executed, i.e., the candidate search point with the minimum template cost obtained (i.e., the candidate search point with the minimum template cost obtained in the most recent iteration) is used as the starting search point (i.e., the five-pointed star position) for the current iteration, and searches for the candidate search point with the minimum template cost to obtain the candidate search point with the minimum template cost in the current iteration.
  • the candidate search point with the minimum template cost obtained i.e., the candidate search point with the minimum template cost obtained in the most recent iteration
  • the starting search point i.e., the five-pointed star position
  • the preset condition can be: the starting search point (i.e., the five-pointed star position) used in the current iteration is the candidate search point with the minimum template cost; or, the preset condition can also be used (or combined) with the maximum number of iterations/maximum number of search points being N (a positive integer).
  • different iterations of the entire search process may use the same search shape.
  • the spacing between candidate search points can be scaled. For example, as the number of iterations increases, after at least one iterative search, the spacing between candidate search points can be proportionally reduced.
  • the three-step search method is shown in Figure 7.
  • search shapes can be combined to form a complete search method, similar to the search method used in motion estimation at the encoder.
  • three examples of actual searches are three examples of actual searches:
  • Example 1 First use the large diamond shown in (a) to perform an iterative search, limiting the maximum number of iterations to 30. After the search is terminated, use the small diamond shown in (e) to search again.
  • Example 2 First use the hexagonal search shown in (d), limit the maximum number of iterations to 30 times, then use the small diamond search shown in (e) once, and then use the search shown in (g) once.
  • Example 3 First use the hexagonal search shown in (d), limit the maximum number of iterations to 40, and then use the search shown in (f) once.
  • the search method used for the current block can be called a template matching-based search method.
  • the template matching-based search method can be the same as or different from the search method used for motion estimation at the encoding end, and this application does not make specific limitations on this.
  • an iterative search is performed on a candidate search point with the minimum template cost to obtain the candidate search point obtained in the current iteration, including:
  • the second candidate search point is ignored, or based on the candidate search points other than the second candidate search point obtained in the second iterative process, the candidate search point with the smallest template cost in the second iterative process is determined.
  • the previous template cost can be loaded instead of being recalculated, thus saving complexity.
  • the overlapping candidate search point is not the search starting point used in the current iterative process (for example, the position of the five-pointed star in the search shape shown in Figure 14), the cost calculation for this overlapping candidate search point can be directly abandoned (because it was not previously selected and cannot be the best), thus saving search complexity.
  • the absolute value of the first offset of any candidate search point among the at least one candidate search point is less than or equal to a third preset threshold, or the absolute value of the second offset of the arbitrary candidate search point is less than or equal to a fourth preset threshold, or the sum of the absolute value of the first offset and the absolute value of the second offset of the arbitrary candidate search point is less than or equal to a fifth preset threshold; wherein the first offset is the offset of the arbitrary candidate search point relative to the first starting search point in the first dimension, and the second offset is the offset of the arbitrary candidate search point relative to the first starting search point in the second dimension.
  • the first dimension is a horizontal direction
  • the second dimension is a vertical direction
  • the absolute value of the offset of any candidate search point in the horizontal and vertical directions relative to the first starting search point is not greater than M (a positive integer) and N (a positive integer), respectively.
  • M and N are 8 integer pixels, respectively.
  • ) does not exceed M (a positive integer), for example, M is set to 10 integer pixels.
  • the present application may not limit the search range or may limit it.
  • the search process is constrained to be carried out only within the limited search range, which can reduce complexity and hardware implementation costs.
  • the method 400 before obtaining the second vector, the method 400 further includes:
  • the cost weight of the first template region is positively correlated with a first size, where the first size is the length of a side of the current block adjacent to the first template region;
  • the cost weight of the first template region is negatively correlated with a first distance, where the first distance is the distance between the first template region and the current block.
  • the prediction device determines cost weights of the multiple template areas based on the current block; based on the cost weights of the multiple template areas, weights the costs of the templates of the multiple template areas relative to any candidate search point in the at least one candidate search point to obtain the template cost of the arbitrary candidate search point; wherein the cost weight of the first template area is positively correlated with the first size.
  • FIG15 is an example of multiple template areas provided in an embodiment of the present application.
  • the multiple template regions of the current block can be weighted differently according to the size of the current block.
  • the prediction device determines the cost weights of the multiple template areas based on the multiple template areas of the current block; based on the cost weights of the multiple template areas, weights the costs of the templates of the multiple template areas relative to any candidate search point in the at least one candidate search point to obtain the template cost of the arbitrary candidate search point; wherein the cost weight of the first template area is negatively correlated with the first distance.
  • FIG16 is another example of multiple template areas provided in an embodiment of the present application.
  • the template area of the current block can be weighted differently based on the distance from the current block. For example, as shown in (a) or (b) of Figure 16, assuming that the template of the current block is divided into four template areas (also called four types of template areas or four sub-areas) based on the distance from the current block, the costs of different template areas can be calculated separately and then weighted. The closer the distance to the current block, the greater the cost weight. For example, the cost weights can be set in descending order of distance to (1, 2, 3, 4) or (1, 2, 2, 3).
  • the first size and the first distance mentioned above can also be combined.
  • the width and height are equal, it is equivalent to considering only the first distance.
  • a larger cost weight is set for the template area close to the long side.
  • the S430 includes any one of the following:
  • the at least one vector includes at least one of the following:
  • a third starting search point is determined based on the fifth vector, and a sixth vector is obtained by searching within the search area using the template of the current block based on the third starting search point.
  • the vector other than the second vector searched based on the first starting search point may be one or more suboptimal candidate search points other than the first optimal search point selected in order of template cost from small to large from the at least one candidate search point mentioned above, and one or more vectors determined based on the one or more suboptimal candidate search points.
  • the method for determining the vector other than the fourth vector based on the search of the second starting search point and the method for determining the vector other than the second vector based on the search of the first starting search point may be the same or different, and this application does not make any specific limitations on this.
  • the prediction device may determine the first prediction block pointed to by the second vector as the prediction block for the current block. Equivalently, the prediction device may obtain the second vector by performing template matching optimization on the first vector, and determine the first prediction block pointed to by the second vector as the prediction block for the current block. For example, if the first VP only includes the first vector, the prediction device may determine the first prediction block pointed to by the second vector as the prediction block for the current block.
  • the prediction device may fuse the first prediction block pointed to by the second vector and at least one of the following prediction blocks to obtain the prediction block of the current block: the prediction block pointed to by a vector other than the second vector searched based on the first starting search point, the prediction block pointed to by the third vector, the prediction block pointed to by the fourth vector, the prediction block pointed to by a vector other than the fourth vector searched based on the second starting search point, the prediction block pointed to by the fifth vector, and the prediction block pointed to by the sixth vector.
  • the prediction device can fuse the first prediction block and the prediction block pointed to by the fourth vector to obtain the prediction block of the current block.
  • the prediction device can first obtain the second vector by performing template matching optimization on the first vector, and obtain the fourth vector by performing template matching optimization on the third vector, and then fuse the first prediction block and the prediction block pointed to by the fourth vector to obtain the prediction block of the current block.
  • the prediction device can fuse the first prediction block and the prediction block pointed to by the fourth vector to obtain the prediction block of the current block.
  • the fusion weight of the first prediction block is equal to the fusion weight of the prediction block pointed to by the fourth vector, or the fusion weight of the first prediction block and the fusion weight of the prediction block pointed to by the fourth vector are negatively correlated with the template cost.
  • the fusion weight of the first prediction block is negatively correlated with the cost of the template of the first prediction block
  • the fusion weight of the prediction block pointed to by the fourth vector is negatively correlated with the cost of the template of the prediction block pointed to by the fourth vector.
  • the at least one vector is indicated by an identifier, and the identifier is carried by a code stream or determined by derivation at a decoding end.
  • the prediction device may determine the at least one vector based on the size of the current block, the number of vectors included in the first VP, or other information.
  • the method 400 further includes at least one of the following:
  • searching for a second best search point with the minimum template cost based on the best search point with the maximum template cost between the second best search point and the first best search point, searching for a third best search point with the minimum template cost after the template is fused with the first template, and updating the best search point with the maximum template cost to the third best search point;
  • the search template is merged with the template of the first optimal search point to obtain a candidate search point with the minimum template cost, thereby obtaining the second optimal search point;
  • the first template is the template of the best search point with the smallest template cost between the first best search point and the second best search point
  • the first best search point is the candidate search point with the smallest template cost searched based on the first starting search point.
  • the template cost after fusion with the first template refers to the difference between the fused template after fusion of the searched template and the first template and the template of the current block.
  • the template cost after fusion with the template of the first optimal search point refers to the difference between the fused template after fusion of the searched template and the template of the first optimal search point and the template of the current block.
  • the prediction device can optimize the two motion vectors separately using template matching. Then, based on the two predicted blocks, bidirectional motion vector prediction is used to fuse them, typically using weighted processing. Specifically, the two motion vectors are first optimized using template matching. The template costs of the two motion vectors are then compared. If the template costs are unequal, a coupled search is performed. The optimized motion vector (MV1) with the lower cost is fixed unchanged (i.e., template P1 remains unchanged). Template matching is then used to search for the best matching template (P2') based on the optimized motion vector (MV2) with the higher cost.
  • MV1 template costs of the two motion vectors
  • the cost is no longer based on the difference between the template and the matching template. Instead, the cost is calculated as the difference between the fused template of P1 and P2' and the template of the current block.
  • the search method is the same or similar to the aforementioned search method, ultimately obtaining the bidirectional motion vectors (MV1, MV2') optimized through the coupled search.
  • the S410 includes:
  • the first starting search point is determined based on the first vector when at least one of the following conditions is met:
  • a second identifier indicates that a vector of the current block is searched using a template of the current block, and the second identifier is carried by a bitstream or determined by deduction at a decoding end;
  • the third identifier carried in the code stream indicates that the current image sequence, current image or current slice to which the current block belongs allows searching for the vector of the current block using the template of the current block;
  • the prediction mode of the current block is an inter prediction mode or an intra block copy prediction mode
  • the current prediction mode is an inter-frame prediction mode
  • the reference frame list of the current block includes a forward reference frame and a backward reference frame.
  • the prediction device determines the first starting search point based on the first vector; when the current prediction mode is an inter-frame prediction mode and the reference frame list of the current block includes only a forward reference frame, the prediction device does not trigger the process of determining the first starting search point based on the first vector.
  • the prediction device when the current prediction mode is an inter-frame prediction mode and the reference frame list of the current block includes a forward reference frame and a backward reference frame, the prediction device optimizes the first vector based on template matching; when the current prediction mode is an inter-frame prediction mode and the reference frame list of the current block includes only a forward reference frame, the prediction device prohibits optimizing the first vector.
  • the third identifier is carried by the code stream or determined by deduction at the decoding end.
  • the S420 includes:
  • the parameter information includes at least one of the following:
  • the maximum number of iterations used in iterative search the maximum number of search points used in iterative search, the search method used in iterative search, and the calculation method of template cost.
  • the size of the serial numbers of the processes involved above does not mean the order of execution.
  • the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
  • FIG17 is a schematic block diagram of a prediction device 500 based on template matching according to an embodiment of the present application.
  • the prediction device 500 includes:
  • a first determining unit 510 is configured to determine a first starting search point based on a first vector included in a first vector prediction VP in a vector prediction VP list of a current block;
  • a search unit 520 is configured to search for a second vector of the current block in a reference area of the current block based on the first starting search point and using a template formed by reconstructed pixels around the current block;
  • the second determining unit 530 is configured to determine a prediction block of the current block based on the second vector.
  • the first VP is indicated by a first identifier, which is carried by a bitstream or determined by derivation at a decoding end; or, the first VP is a VP with the smallest template cost in the VP list.
  • the codeword length of the first identifier is positively correlated with the template cost of the first VP.
  • the first determining unit 510 before determining the first starting search point, is further configured to:
  • At least one of deduplication, vector adjustment, template cost evaluation, and template cost-based sorting is performed on the initial VPs in the initial list to obtain the VP list.
  • the first determining unit 510 is specifically configured to:
  • At least one dimension of the first vector is adjusted to obtain an adjusted vector of the first vector, and a position pointed to by the adjusted vector is determined as the first starting search point.
  • the first determining unit 510 is specifically configured to perform at least one of the following:
  • At least one dimension of the first vector is offset to obtain the adjustment vector.
  • the search unit 520 is specifically configured to:
  • the first vector is adjusted to point to a first optimal search point with the minimum template cost among the at least one candidate search point, to obtain the second vector.
  • the search unit 520 is specifically configured to perform at least one of the following:
  • the precondition includes at least one of the following:
  • the candidate search point with the minimum template cost obtained in the current iteration process is used as the starting search point used in the current iteration process;
  • the number of iterations is greater than or equal to the maximum number of iterations
  • the number of points searched is greater than or equal to the maximum number of search points.
  • the search unit 520 is specifically configured to:
  • the search method includes at least one search shape and an iteration number corresponding to each search shape in the at least one search shape, and the at least one search shape includes the first search shape.
  • the search unit 520 is specifically configured to:
  • the second candidate search point is ignored, or based on the candidate search points other than the second candidate search point obtained in the second iterative process, the candidate search point with the smallest template cost in the second iterative process is determined.
  • an absolute value of a first offset of any candidate search point among the at least one candidate search point is less than or equal to a third preset threshold, or an absolute value of a second offset of the any candidate search point is less than or equal to a fourth preset threshold, or a sum of an absolute value of the first offset and an absolute value of the second offset of the any candidate search point is less than or equal to a fifth preset threshold;
  • the first offset is the offset of the arbitrary candidate search point relative to the first starting search point in the first dimension
  • the second offset is the offset of the arbitrary candidate search point relative to the first starting search point in the second dimension
  • the search unit 520 before obtaining the second vector, is further configured to:
  • the cost weight of the first template region among the multiple template regions satisfies at least one of the following:
  • the cost weight of the first template region is positively correlated with a first size, where the first size is the length of a side of the current block adjacent to the first template region;
  • the cost weight of the first template region is negatively correlated with a first distance, where the first distance is the distance between the first template region and the current block.
  • the second determining unit 530 is specifically configured to perform any of the following:
  • the at least one vector includes at least one of the following:
  • a third starting search point is determined based on the fifth vector, and a sixth vector is obtained by searching within the search area using the template of the current block based on the third starting search point.
  • the search unit 520 is further configured to perform at least one of the following:
  • searching for a second best search point with the minimum template cost based on the best search point with the maximum template cost between the second best search point and the first best search point, searching for a third best search point with the minimum template cost after the template is fused with the first template, and updating the best search point with the maximum template cost to the third best search point;
  • the search template is merged with the template of the first optimal search point to obtain a candidate search point with the minimum template cost, thereby obtaining the second optimal search point;
  • the first template is the template of the best search point with the smallest template cost between the first best search point and the second best search point
  • the first best search point is the candidate search point with the smallest template cost searched based on the first starting search point.
  • the first determining unit 510 is specifically configured to:
  • the first starting search point is determined based on the first vector when at least one of the following conditions is met:
  • a second identifier indicates that a vector of the current block is searched using a template of the current block, and the second identifier is carried by a bitstream or determined by deduction at a decoding end;
  • the third identifier carried in the code stream indicates that the current image sequence, current image or current slice to which the current block belongs allows searching for the vector of the current block using the template of the current block;
  • the prediction mode of the current block is an inter prediction mode or an intra block copy prediction mode
  • the current prediction mode is an inter-frame prediction mode
  • the reference frame list of the current block includes a forward reference frame and a backward reference frame.
  • the search unit 520 is specifically configured to:
  • the parameter information includes at least one of the following:
  • the maximum number of iterations used in iterative search the maximum number of search points used in iterative search, the search method used in iterative search, and the calculation method of template cost.
  • prediction device 500 provided in the embodiment of the present application may correspond to the execution entity in the method embodiment of the present application, and the various units in the prediction device 500 are respectively for implementing the corresponding processes of the prediction method 400 shown in Figure 11. For the sake of brevity, they will not be repeated here.
  • the prediction device provided in the embodiment of the present application can implement each process in the method embodiment of Figure 11 and achieve the same technical effect. To avoid repetition, it will not be described here.
  • the various units in the prediction device 500 involved in the embodiment of the present application are divided based on logical functions.
  • the function of a unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit, or even, these functions can also be implemented with the assistance of one or more other units.
  • part or all of the prediction device 500 is merged into one or several other units.
  • a certain (some) unit in the prediction device 500 can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the prediction device 500 can also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
  • module or unit involved in the embodiments of the present application refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof.
  • a processor or multiple processors or memories can be used to implement one or more modules or units.
  • each module or unit can be part of an overall module or unit that includes the function of the module or unit.
  • the prediction device 500 involved in the embodiment of the present application can be constructed by running a computer program (including program code) capable of executing each step involved in the corresponding method on a general-purpose computing device of a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM), and implementing the method of the embodiment of the present application.
  • the computer program can be recorded on, for example, a computer-readable storage medium and loaded into an electronic device via the computer-readable storage medium, and the computer program is used to implement the corresponding method of the embodiment of the present application.
  • the units involved above can be implemented in hardware form, or can be implemented by instructions in the form of software, or can be implemented in the form of a combination of software and hardware.
  • the steps of the method embodiment in the embodiment of the present application can be completed by the integrated logic circuit of the hardware in the processor and/or the instructions in the form of software, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software in a decoding processor.
  • the software can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc.
  • the software in the memory can be executed by the processor to complete the steps in the above-mentioned method embodiments.
  • FIG18 is a schematic structural diagram of an electronic device 600 provided in this application.
  • the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620.
  • the processor 610 and the computer-readable storage medium 620 may be connected via a bus or other means.
  • the computer-readable storage medium 620 is used to store a computer program 621, which includes computer instructions.
  • the processor 610 is used to execute the computer instructions stored in the computer-readable storage medium 620.
  • the processor 610 is the computing core and control core of the electronic device 600 and is suitable for implementing one or more computer instructions, specifically loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.
  • the processor 610 may also be referred to as a central processing unit (CPU).
  • the processor 610 may include, but is not limited to, a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, discrete hardware components, and the like.
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA field programmable gate array
  • the computer-readable storage medium 620 may be a high-speed RAM memory or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it may be at least one computer-readable storage medium located away from the aforementioned processor 610.
  • the computer-readable storage medium 620 includes, but is not limited to, volatile memory and/or non-volatile memory.
  • the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.
  • the volatile memory may be a random access memory (RAM), which is used as an external cache.
  • RAM random access memory
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • ESDRAM enhanced synchronous dynamic random access memory
  • SLDRAM synchronous link DRAM
  • DR RAM direct RAM bus random access memory
  • the electronic device 600 may further include a transceiver 630 .
  • the processor 610 may control the transceiver 630 to communicate with other devices. Specifically, the processor 610 may send information or data to other devices or receive information or data sent by other devices.
  • the transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include one or more antennas.
  • the electronic device 600 can be the prediction device 500 involved in the embodiment of the present application; the computer-readable storage medium 620 stores computer instructions; the processor 610 loads and executes the computer instructions stored in the computer-readable storage medium 620 to implement the corresponding steps in the prediction method 400 provided in the present application; in other words, the computer instructions in the computer-readable storage medium 620 are loaded by the processor 610 and execute the corresponding steps in the prediction method 400; to avoid repetition, they are not repeated here.
  • the present application also provides a coding and decoding system, including the encoder and decoder mentioned above.
  • the present application also provides a computer-readable storage medium (Memory), which stores computer instructions.
  • a computer-readable storage medium (Memory)
  • the computer device executes the prediction method mentioned above.
  • the computer-readable storage medium is a memory device in a decoder or encoder, which is used to store programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices.
  • the computer-readable storage medium can be used to provide storage space, and the storage space can store the operating system of the electronic device.
  • one or more computer instructions suitable for being loaded and executed by the processor are also stored in the storage space, for example, one or more computer instructions for executing the prediction method mentioned above are stored, and these computer instructions can be one or more computer programs (including program codes).
  • a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium.
  • a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer to perform the prediction methods provided in the various optional embodiments described above.
  • the computer device involved in the present application can be any device or apparatus capable of performing data processing, for example, including but not limited to: a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions involved in the present application can be stored in a computer-readable storage medium, or can be transmitted between one computer-readable storage medium and another computer-readable storage medium.
  • the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means.
  • the present application further provides a code stream, which can be obtained by decoding the code stream using the prediction method provided by the present application, or generating the code stream using the prediction method provided by the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请提供了一种基于模板匹配的预测方法、装置、设备及存储介质，其涉及编解码技术领域，该方法包括：基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量；基于该第二矢量，确定该当前块的预测块。由于编码端和解码端都可以采用该方法确定该当前块的预测块，即编码端可以不传递运动信息到解码端，可以节省运动信息传递时所占用的码率，进而能够提升码率。

Description

基于模板匹配的预测方法、装置、设备及存储介质
本申请要求于2024年03月15日提交中国专利局、申请号为202410306896.X、发明名称为“基于模板匹配的预测方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及编解码技术领域，并且更具体地，涉及一种基于模板匹配的预测方法、装置、设备及存储介质。
背景技术
截止目前，帧间预测模式或者帧内块复制模式，都是通过模板匹配的方式在参考区域中搜素与当前编码块最接近的图像块，然后将矢量信息(例如，用于指示预测块的位置的运动矢量)编码发送到解码端，以便解码端通过矢量信息在参考区域获取得到当前块的预测块。
然而，将矢量信息编码发送到解码端，需要消耗码率去传送矢量信息，进而降低了码率。
发明内容
本申请实施例提供了一种基于模板匹配的预测方法、装置、设备及存储介质，能够提升码率。
第一方面，本申请实施例提供了一种基于模板匹配的预测方法，包括：
基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；
基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量；
基于该第二矢量，确定该当前块的预测块。
第二方面，本申请实施例提供了一种基于模板匹配的预测装置，包括：
第一确定单元，用于基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；
搜索单元，用于基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量；
第二确定单元，用于基于该第二矢量，确定该当前块的预测块。
第三方面，本申请实施例提供了一种电子设备，包括：
处理器，适于实现计算机指令；以及，
计算机可读存储介质，计算机可读存储介质存储有计算机指令，计算机指令适于由处理器加载并执行上文涉及的第一方面的方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，该计算机可读存储介质存储有计算机指令，该计算机指令被计算机设备的处理器读取并执行时，使得计算机设备执行上文涉及的第一方面的方法。
第五方面，本申请实施例提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上文涉及的第一方面的方法。
第六方面，本申请实施例提供了一种码流，该码流如上文涉及的第一方面所述的方法生成的码流。
本申请实施例中，该基于模板匹配的预测方法包括：基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量；基于该第二矢量，确定该当前块的预测块。相当于，该基于模板匹配的预测方法可以基于该第一起始搜索点，借助当前块周围已重建像素构建模板，在该当前块的参考区域内搜索该当前块的第二矢量，进而基于该第二矢量，确定该当前块的预测块。由于编码端和解码端都可以采用该基于模板匹配的预测方法确定该当前块的预测块，即编码端可以不需要传递矢量信息到解码端，可以节省运动信息传递时所占用的码率，进而能够提升码率。
此外，基于该第一起始搜索点搜索得到第二矢量，进而基于该第二矢量确定该当前块的预测块，相当于对该第一矢量进行了优化，能够获得较高质量的预测块，进而提升了该当前块的预测效果。
附图说明
图1为本申请实施例涉及的一种视频编解码系统的示意性框图。
图2是本申请实施例涉及的视频编码器的示意性框图。
图3是本申请提供的编码树单元和编码单元的关系的示意性结构图。
图4是本申请实施例涉及的视频解码器的示意性框图。
图5是本申请实施例提供的整像素和分像素的示例。
图6是本申请实施例提供的空域MVP的示例。
图7是本申请实施例提供的三步搜索方法的示例。
图8是本申请实施例提供的四步搜索方法的示例。
图9是本申请实施例提供的菱形搜索方法的示例。
图10是本申请实施例提供的六边形搜索方法的示例。
图11是本申请实施例提供的基于模板匹配的预测方法的示意性流程图。
图12是本申请实施例提供的模板的示例。
图13是本申请实施例提供的搜索点的模板和当前块的模板的示例。
图14是本申请实施例提供的搜索方法包括的搜索形状的示例。
图15是本申请实施例提供的多个模板区域的示例。
图16是本申请实施例提供的多个模板区域的另一示例。
图17是本申请实施例提供的基于模板匹配的预测装置的示意性框图。
图18是本申请实施例提供的电子设备的示意性框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚描述。
本申请提供的方案涉及数字压缩技术领域。
其中，数字视频压缩技术主要是将庞大的数字影像视频数据进行压缩，以便于传输以及存储等。
本申请提供的方案可应用数字视频编码技术领域。
其中，数字视频编码技术领域包括但不限于以下中的至少一项：图像编解码领域、视频编解码领域、硬件视频编解码领域、专用电路视频编解码领域以及实时视频编解码领域。此外，本申请提供的方案可结合到以下标准：音视频编码标准(Audio Video coding Standard，AVS)、第二代AVS标准(AVS2)或第三代AVS标准(AVS3)。例如，包括但不限于：H.264/音视频编码(Audio Video coding，AVC)标准、H.265/高效视频编码(High Efficiency Video Coding，HEVC)标准以及H.266/多功能视频编码(Versatile Video Coding，VVC)标准。另外，本申请提供的方案可以用于对图像进行有损压缩(lossy compression)，也可以用于对图像进行无损压缩(lossless compression)。其中，该无损压缩可以是视觉无损压缩(visually lossless compression)，也可以是数学无损压缩(mathematically lossless compression)。
为便于理解本申请提供的技术方案，下面对相关术语进行说明。
立即解码刷新(Instantaneous Decoding Refresh，IDR)帧：在视频编码算法中，图像以序列为单位进行组织。一个序列的第一个图像叫做IDR帧，IDR图像都是I帧。IDR帧也称为IDR图像。
I帧：IDR帧会导致参考帧列表(Decoded Picture Buffer，DPB)清空，而I帧不会。IDR图像一定是I帧，但I帧不一定是IDR帧。一个序列中可以有很多的I帧，I帧之后的帧可以引用I帧之间的帧做运动参考。
P帧：前向预测编码帧。P帧表示的是这一帧跟之前的一个关键帧(或P帧)的差别，解码时需要用之前缓存的画面叠加上本帧定义的差别，生成最终画面。
B帧：双向预测内插编码帧。B帧是双向差别帧，也就是B帧记录的是本帧与前后帧的差别，B帧可以作其它B帧的参考帧，也可以不作为其它B帧参考帧。
量化参数(Quantization Parameter，QP)：码率控制算法主要是通过调整离散余弦变换的量化参数大小输出目标码率。实际上量化参数(QP)反映了空间细节压缩情况，如QP小，大部分的细节都会被保留；QP增大，一些细节丢失，码率降低，图像失真加强和质量下降。也就是说，QP和码率成反比的关系，而且随着视频源复杂度的提高，这种反比关系会更明显。
帧内预测：预测块是基于已编码重建块和当前块形成的块。
帧间预测：主要包括运动估计(运动搜索方法、运动估计准则、亚像素插值和运动矢量估计)和运动补偿，是图像组(Group Of Pictures，GOP)粒度时序上的参考和预测插值补偿。
绝对差和(Sum of Absolute Difference，SAD)：即绝对差值的和。
绝对误差和(Sum of Absolute Error，SAE)：即绝对误差的和。
绝对变换差异之和(Sum of Absolute Transformed Difference，SATD)：例如哈达玛(hadamard)变换后再绝对值求和。
运动补偿(Motion Compensation，MC)：通过预测图像中物体的运动轨迹并补偿其位移，使得连续图像之间的数据量得以减少。
运动估计(Motion Estimation，ME)：通过寻找相邻图像之间的相关性以及计算块之间的运动矢量，去除时间维度的冗余信息，进一步提高压缩效率。
BD-rate：评价视频编码算法性能的主要参数之一,表示新算法编码的视频相对于原来的算法在码率和峰值信噪比(Peak Signal-to-Noise Ratio，PSNR)上的变化情况。
率失真优化(Rate Distortion Optimation，RDO)：编码过程中有许多的模式可以选择，有些模式的图像失真较小，但是码率却很大；有些模式的图像失真较大，但是码率却很小。基于此可以在不超过某最大码率的情况下，失真达到最小。在视频编码的率失真优化中，可以将最大码率作为约束条件，失真作为优化目标，然后使用拉格朗日乘子法来寻找最优的编码参数。
宏块：编码的基本单位，一个图像首先要划分成宏块才能进行处理。例如，可以按照一定的大小为单位(H.264中按照16×16的大小)将图像划分为宏块，编码过程以块为单位进行编码。
参考图像：在视频编解码中，一种作为其他图像的参考数据，用于其他图像在编码/解码过程中获取图像间的参考数据的重建图像。
运动矢量(Motion Vector，MV)：一个二维向量，用于描述编码器中一个编码块从其所在位置运动到另一个位置时，所发生的位置偏移。
运动搜索(Motion Search，MS)：为当前编码块在参考图像中按照某种算法寻找最优匹配块的过程。
预测运动矢量(Motion Vector Prediction，MVP)：根据已有的信息，对当前MV按照一定的算法进行的预测。
需要说明的是，本申请的实施方式部分使用的术语仅用于对本申请的实施例进行解释，而非旨在限定本申请。
例如，本文中的术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。术语“至少一项”，仅仅是一种描述列举对象的组合关系，表示可以存在一项或多项，例如，以下中的至少一项：A、B、C，可以表示以下几种组合情况：单独存在A，单独存在B，单独存在C，同时存在A和B，同时存在A和C，同时存在B和C，同时存在A、B以及C。术语“多个”是指两个或两个以上。字符“/”，一般表示前后关联对象是一种“或”的关系。
再如，术语“对应”可表示两者之间具有直接对应或间接对应的关系，也可以表示两者之间具有关联关系，也可以是指示与被指示、配置与被配置等关系。术语“指示”可以是直接指示，也可以是间接指示，还可以是表示具有关联关系。举例说明，A指示B，可以表示A直接指示B，例如B可以通过A获取；也可以表示A间接指示B，例如A指示C，B可以通过C获取；还可以表示A和B之间具有关联关系。术语“预定义”或“预配置”可以在设备中预先保存相应的代码、表格或其他可用于指示的相关信息，也可以是指由协议约定。“协议”可以指本领域的标准协议。术语“在……时”可以被解释成为“如果”或“若”或“当……时”或“响应于”等类似描述。类似地，取决于语境，短语“如果确定”或“如果检测(陈述的条件或事件)”可以被解释成为“当确定时”或“响应于确定”或“当检测(陈述的条件或事件)时”或“响应于检测(陈述的条件或事件)”等类似描述。术语“第一”、“第二”、“第三”、“第四”、“第A”、“第B”等是用于区别不同对象，而不是用于描述特定顺序。术语“包括”和“具有”以及它们任何变形，意图在于覆盖不(或非)排他的包含。其中，数字视频压缩技术主要是将庞大的数字影像视频数据进行压缩，以便于传输以及存储等。
为了便于理解，首先结合图1对本申请实施例涉及的视频编解码系统进行介绍。
图1为本申请实施例涉及的一种视频编解码系统的示意性框图。
如图1所示，该视频编解码系统100包含编码设备110和解码设备120。
其中，编码设备110用于对视频数据进行编码(可以理解成压缩)产生码流，并将码流传输给解码设备120。解码设备120对编码设备110编码产生的码流进行解码，得到解码后的视频数据。
编码设备110可以理解为具有视频编码功能的设备，解码设备120可以理解为具有视频解码功能的设备，即本申请实施例对编码设备110和解码设备120包括更广泛的装置，例如包含智能手机、台式计算机、移动计算装置、笔记本(例如，膝上型)计算机、平板计算机、机顶盒、电视、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机等。
编码设备110可以经由信道130将编码后的视频数据(例如码流)传输给解码设备120。
信道130可以包括能够将编码后的视频数据从编码设备110传输到解码设备120的一个或多个媒体和/或装置。信道130可以包括使编码设备110能够实时地将编码后的视频数据直接发射到解码设备120的一个或多个通信媒体。编码设备110可根据通信标准来调制编码后的视频数据，且将调制后的视频数据发射到解码设备120。其中通信媒体包含无线通信媒体，例如射频频谱。通信媒体还可以包含有线通信媒体，例如一根或多根物理传输线。信道130可以包括存储介质，该存储介质可以存储编码设备110编码后的视频数据。存储介质包含多种本地存取式数据存储介质，例如光盘、DVD、快闪存储器等。在该实例中，解码设备120可从该存储介质中获取编码后的视频数据。信道130可以包含存储服务器，该存储服务器可以存储编码设备110编码后的视频数据。在此实例中，解码设备120可以从该存储服务器中下载存储的编码后的视频数据。可选的，该存储服务器可以存储编码后的视频数据且可以将该编码后的视频数据发射到解码设备120，例如web服务器(例如，用于网站)、文件传送协议(FTP)服务器等。
编码设备110包含视频编码器112及输出接口113。其中，输出接口113可以包含调制器/解调器(调制解调器)和/或发射器。视频编码器112经由输出接口113将编码后的视频数据直接传输到解码设备120。编码后的视频数据还可存储于存储介质或存储服务器上，以供解码设备120后续读取。
编码设备110除了包括视频编码器112和输入接口113外，还可以包括视频源111。视频源111可包含视频采集装置(例如，视频相机)、视频存档、视频输入接口、计算机图形系统中的至少一个，其中，视频输入接口用于从视频内容提供者处接收视频数据，计算机图形系统用于产生视频数据。视频编码器112对来自视频源111的视频数据进行编码，产生码流。视频数据可包括一个或多个图像(picture)或图像序列(sequence of pictures)。码流以比特流的形式包含了图像或图像序列的编码信息。编码信息可以包含编码图像数据及相关联数据。相关联数据可包含序列参数集(sequence parameter set，SPS)、图像参数集(picture parameter set，PPS)及其它语法结构。SPS可含有应用于一个或多个序列的参数。PPS可含有应用于一个或多个图像的参数。语法结构是指：码流中以指定次序排列的零个或多个语法元素的集合。
解码设备120包含输入接口121和视频解码器122。输入接口121可包含接收器及/或调制解调器。其中，输入接口121可通过信道130接收编码后的视频数据。视频解码器122用于对编码后的视频数据进行解码，得到解码后的视频数据。
解码设备120除包括输入接口121和视频解码器122外，还可以包括显示装置123。视频解码器122得到解码后的视频数据后，将解码后的视频数据传输到显示装置123。显示装置123显示解码后的视频数据。显示装置123可与解码设备120整合或在解码设备120外部。显示装置123可包括多种显示装置，例如液晶显示器(LCD)、等离子体显示器、有机发光二极管(OLED)显示器或其它类型的显示装置。
应当理解，图1仅为本申请的示例，不应理解为对本申请的显示，也即是说，本申请实施例的技术方案不限于图1所示的系统框架，例如本申请的技术还可以应用于单侧的视频编码或者单侧的视频解码。
下面对本申请实施例涉及的视频编码框架进行介绍。
图2是本申请实施例涉及的视频编码器200的示意性框图。
应当理解，该视频编码器200可应用于亮度色度(YCbCr，YUV)格式的图像数据上。例如，YUV比例可以为4:2:0、4:2:2或者4:4:4，Y表示明亮度(Luma)，Cb(U)表示蓝色色度，Cr(V)表示红色色度，U和V表示为色度(Chroma)用于描述色彩及饱和度。例如，在颜色格式上，4:2:0表示每4个像素有4个亮度分量，2个色度分量(YYYYCbCr)，4:2:2表示每4个像素有4个亮度分量，4个色度分量(YYYYCbCrCbCr)，4:4:4表示全像素显示(YYYYCbCrCbCrCbCrCbCr)。当然，也可以应用于红绿蓝(Red-Green-Blue，RGB)格式的图像数据，本申请对此不作具体限定。
视频编码器200读取视频流后，针对视频流中的每帧图像，可将其划分成若干个编码树单元(coding tree unit，CTU)。在一些例子中，CTU可被称作树块、最大编码单元(Largest Coding unit，LCU)、或编码树区块(coding tree block，CTB)。一个CTU大小例如可以为128×128、64×64、32×32等。
图3是本申请提供的编码树单元和编码单元的关系的示意性结构图。
如图3所示，一个CTU又可以继续被划分成若干个编码单元(Coding Unit，CU)进行编码，CU可以为矩形块也可以为方形块。CU可以进一步划分为预测单元(prediction Unit，PU)和变换单元(transform unit，TU)，进而使得编码、预测、变换分离，处理的时候更灵活。在一种示例中，CTU以树(例如四叉树)方式划分为CU，CU以树方式(例如四叉树)划分为TU、PU。
视频编码器及视频解码器可支持各种PU大小。
假定特定CU的大小为2N×2N，视频编码器及视频解码器可支持2N×2N或N×N的PU大小以用于帧内(intra)预测，且支持2N×2N、2N×N、N×2N、N×N或类似大小的对称PU以用于帧间(inter)预测。视频编码器及视频解码器还可支持2N×nU、2N×nD、nL×2N及nR×2N的不对称PU以用于帧间(inter)预测。
如图2所示，该视频编码器200可包括：预测单元210、残差单元220、变换/量化单元230、反变换/量化单元240、重建单元250、环路滤波单元260、解码图像缓存270和熵编码单元280。需要说明的是，视频编码器200可包含更多、更少或不同的功能组件。在本申请中，当前块(current block)可以称为当前编码单元(CU)或当前预测单元(PU)等。预测块也可称为预测图像块或图像预测块，重建图像块也可称为重建块或图像重建图像块。
预测单元210包括帧间(inter)预测单元211、帧内(intra)预测单元212。由于视频中一个图像中的相邻像素之间存在很强的相关性，在视频编解码技术中，使用帧内预测的方法消除相邻像素之间的空间冗余。由于视频中的相邻图像之间存在着很强的相似性，使用帧间预测方法消除相邻图像之间的时间冗余，从而提高编码效率。
帧间预测单元211可用于帧间预测，其可以包括运动估计(motion estimation)和运动补偿(motion compensation)，可以参考不同帧的图像信息，帧间预测使用运动信息，从参考帧中，找到参考块，根据参考块生成预测块，用于消除时间冗余；该参考帧可以为P帧和/或B帧，P帧指：向前预测帧，B帧指：双向预测帧。帧间预测使用运动信息找到参考块后，根据参考块生成预测块。运动信息包括参考帧所属的帧列表、帧索引以及运动矢量。运动矢量可以是整像素的或者是分像素的，如果运动矢量是分像素的，那么需要在参考帧中，使用插值滤波做出所需的分像素的块，参考块就是根据运动矢量找到的整像素或者分像素的块。有的技术会直接把参考块作为预测块，有的技术会在参考块的基础上再处理生成预测块。在参考块的基础上再处理生成预测块也可以理解为把参考块作为预测块然后再在预测块的基础上处理生成新的预测块。
帧内预测单元212只参考同一帧图像的信息，预测当前码图像块内的像素信息，用于消除空间冗余。帧内预测所使用的参考帧可以为I帧。
帧内预测有多种预测模式，可借助角度预测模式与非角度(Non-angle)预测模式对待编码图像块进行预测，以得到预测块，根据预测块与待编码图像块，计算得到的率失真信息，筛选出待编码图像块最优的预测模式，并将该预测模式写入码流以传输到解码端。解码端解析出预测模式，预测得到目标解码块的预测块并叠加基于码流而获取的时域残差块，可得到重建块。
以国际数字视频编码标准H系列为例，H.264/AVC标准有8种角度预测模式和1种非角度(Non-angle)预测模式，H.265/HEVC扩展到33种角度预测模式和2种非角度(Non-angle)预测模式。HEVC使用的帧内预测模式有平面模式(Planar)、直流(DC)和33种角度模式，共35种预测模式。VVC使用的帧内模式有Planar、DC和65种角度模式，共67种预测模式，其包括传统预测模式和非传统的预测模式，非传统的预测模式可以包括矩阵加权帧内预测(Matrix weighted intra-frame prediction，MIP)模式。传统预测模式包括：模式编号0的平面(planar)模式、模式编号1的DC模式和模式编号2到模式编号66的角度预测模式。需要说明的是，随着角度模式的增加，帧内预测的预测结果将会更加精确，也更加符合对高清以及超高清的数字视频发展的需求，上述帧内预测模式仅为本申请的示例，不应对本申请产生限定。
残差单元220可基于CU的像素块及CU的PU的预测块来产生CU的残差块。举例来说，残差单元220可产生CU的残差块，使得残差块中的每一采样具有等于以下两者之间的差的值：CU的像素块中的采样，及CU的PU的预测块中的对应采样。
变换/量化单元230可量化变换系数。变换/量化单元230可基于与CU相关联的量化参数(QP)值来量化与CU的TU相关联的变换系数。视频编码器200可通过调整与CU相关联的QP值来调整应用于与CU相关联的变换系数的量化程度。
反变换/量化单元240可分别将逆量化及逆变换应用于量化后的变换系数，以从量化后的变换系数重建残差块。
重建单元250可将重建后的残差块的采样加到预测单元210产生的一个或多个预测块的对应采样，以产生与TU相关联的重建图像块。通过此方式重建CU的每一个TU的采样块，视频编码器200可重建CU的像素块。
环路滤波单元260用于对反变换与反量化后的像素进行处理，弥补失真信息，为后续编码像素提供更好的参考，例如可执行消块滤波操作以减少与CU相关联的像素块的块效应。在一些实施例中，环路滤波单元260包括：去块(DeBlocking)滤波(DeBlocking Filter，DBF)单元和样点自适应补偿/自适应环路滤波(SAO/ALF)单元，其中，DBF单元用于去方块效应，SAO/ALF单元用于去除振铃效应。
解码图像缓存270可存储重建后的像素块。其中，帧间预测单元211可使用解码图像缓存270中含有重建后的像素块的参考图像来对其它图像的PU执行帧间预测。另外，帧内预测单元212可使用解码图像缓存270中的重建后的像素块来对在与CU相同的图像中的其它PU执行帧内预测。
熵编码单元280可接收来自变换/量化单元230的量化后的变换系数。熵编码单元280可对量化后的变换系数执行一个或多个熵编码操作以产生熵编码后的数据。
图4是本申请实施例涉及的视频解码器的示意性框图。
如图4所示，视频解码器300包含：熵(entropy)解码单元310、预测单元320、反量化/变换单元330、重建单元340、环路滤波单元350及解码图像缓存360。需要说明的是，视频解码器300可包含更多、更少或不同的功能组件。
其中，熵解码单元310可解析码流以从码流提取语法元素。作为解析码流的一部分，熵解码单元310可解析码流中的经熵编码后的语法元素。预测单元320、反量化/变换单元330、重建单元340及环路滤波单元350可根据从码流中提取的语法元素来解码视频数据，即产生解码后的视频数据。
预测单元320包括：帧内(intra)预测单元322、帧间(inter)预测单元321。帧内预测单元322可执行帧内预测以产生PU的预测块。帧内预测单元322可使用帧内预测模式以基于空间相邻PU的像素块来产生PU的预测块。帧内预测单元322还可根据从码流解析的一个或多个语法元素来确定PU的帧内预测模式。帧间预测单元321可根据从码流解析的语法元素来构造第一参考图像列表(列表0)及第二参考图像列表(列表1)。此外，如果PU使用帧间预测编码，则熵解码单元310可解析PU的运动信息。帧间预测单元321可根据PU的运动信息来确定PU的一个或多个参考块。帧间预测单元321可根据PU的一个或多个参考块来产生PU的预测块。
反量化/变换单元330可逆量化(即，解量化)与TU相关联的变换系数。反量化/变换单元330可使用与TU的CU相关联的QP值来确定量化程度。在逆量化变换系数之后，反量化/变换单元330可将一个或多个逆变换应用于逆量化变换系数，以便产生与TU相关联的残差块。
重建单元340使用与CU的TU相关联的残差块及CU的PU的预测块以重建CU的像素块。例如，重建单元340可将残差块的采样加到预测块的对应采样以重建CU的像素块，得到重建图像块。
环路滤波单元350可执行消块滤波操作以减少与CU相关联的像素块的块效应。
视频解码器300可将CU的重建图像存储于解码图像缓存360中。视频解码器300可将解码图像缓存360中的重建图像作为参考图像用于后续预测，或者，将重建图像传输给显示装置呈现。
结合图2和图4来说，视频编解码的基本流程如下：
在编码端，将一帧图像划分成图像块，针对当前块，预测单元210使用帧内预测或帧间预测，预测当前块(即待编码块)的预测块。残差单元220可基于预测块与当前块(即待编码块)的原始块，计算残差块，即预测块和原始块的差值，该残差块也可称为残差信息。该残差块经由变换/量化单元230变换与量化等过程，可以去除人眼不敏感的信息，以消除视觉冗余。可选的，经过变换/量化单元230变换与量化之前的残差块可称为时域残差块，经过变换/量化单元230变换与量化之后的时域残差块可称为频率残差块或频域残差块。熵编码单元280接收到变化量化单元230输出的量化后的变化系数，可对该量化后的变化系数进行熵编码，输出码流。例如，熵编码单元280可根据目标上下文模型以及二进制码流的概率信息消除字符冗余。
在解码端，熵解码单元310可解析码流得到当前块(即待解码块)的预测信息、量化系数矩阵等，预测单元320基于预测信息，使用帧内预测或帧间预测，预测当前块(即待解码块)的预测块。反量化/变换单元330使用从码流得到的量化系数矩阵，对量化系数矩阵进行反量化、反变换得到残差块。重建单元340将预测块和残差块相加得到重建块。重建块组成重建图像，环路滤波单元350基于图像或基于块对重建图像进行环路滤波，得到解码图像。值得注意的是，编码端同样需要采用和解码器类似的操作获得解码图像。该解码图像也可以称为重建图像，重建图像可以为后续图像，作为进行帧间预测的参考图像。
此外，编码器确定的块划分信息，以及预测、变换、量化、熵编码、环路滤波等模式信息或者参数信息等在必要时携带在码流中。解码端通过解析码流及根据已有信息进行分析确定与编码端相同的块划分信息，预测、变换、量化、熵编码、环路滤波等模式信息或者参数信息，从而保证编码器获得的解码图像和解码器获得的解码图像相同。
需要说明的是，由于并行处理的需要，图像可以被划分成片(slice)等，同一个图像中的片可以并行处理，也就是说它们之间没有数据依赖。术语“帧”可以理解为图像或slice等。上述是基于块的编解码框架下的视频编解码器的基本流程，随着技术的发展，该框架或流程的一些模块或步骤可能会被优化，即本申请不限于该框架及流程。
为便于理解本申请提供的技术方案，下面对相关内容进行说明。
(1)视频编码技术概述。
视频信号，从信号的获取方式看，可以分为摄像机拍摄到的以及计算机生成的两种方式。由于统计特性的不同，其对应的压缩编码方式也可能有所区别。
视频编码技术，以国际视频编码标准HEVC，VVC，以及中国国家视频编码标准AVS为例，采用了混合编码框架，对输入的原始视频信号，进行了如下一系列的操作和处理：
块划分结构(block partition structure)：输入图像根据一个的大小，划分成若干个不重叠的处理单元，每个处理单元将进行类似的压缩操作。这个处理单元被称作CTU，或者LCU。CTU再往下，可以继续进行更加精细的划分，得到一个或多个基本编码的单元，称之为CU。每个CU是一个编码环节中最基本的元素。以下描述的是对每一个CU可能采用的各种编码方式。
预测编码(Predictive Coding)：包括了帧内预测和帧间预测等方式，原始视频信号经过选定的已重建视频信号的预测后，得到残差视频信号。编码端为当前CU决定在众多可能的预测编码模式中，选择最适合的一种，并告知解码端。帧内预测：预测的信号来自于同一图像内已经编码重建过的区域。帧间预测：预测的信号来自已经编码过的，不同于当前图像的其他图像(称之为参考图像)。
变换编码及量化(Transform&Quantization)：残差视频信号经过DFT，DCT等变换操作，将信号转换到变换域中，称之为变换系数。在变换域中的信号，进一步的进行有损的量化操作，丢失一定的信息，使得量化后的信号有利于压缩表达。在一些视频编码标准中，可能有多于一种变换方式可以选择，因此，编码端也需要为当前编码CU选择其中的一种变换，并告知解码端。量化的精细程度通常由量化参数(QP)来决定，QP取值较大，表示更大取值范围的系数将被量化为同一个输出，因此通常会带来更大的失真，及较低的码率；相反，QP取值较小，表示较小取值范围的系数将被量化为同一个输出，因此通常会带来较小的失真，同时对应较高的码率。
熵编码(Entropy Coding)或统计编码：量化后的变换域信号，将根据各个值出现的频率，进行统计压缩编码，最后输出二值化(0或者1)的压缩码流。同时，编码产生其他信息，例如选择的模式，运动矢量等，也需要进行熵编码以降低码率。统计编码是一种无损编码方式，可以有效的降低表达同样的信号所需要的码率。常见的统计编码方式有变长编码(Variable Length Coding，VLC)或者基于上下文的二值化算术编码(Content Adaptive Binary Arithmetic Coding，CABAC)。
环路滤波(Loop Filtering)：已经编码过的图像，经过反量化，反变换及预测补偿的操作(上述2～4的反向操作)，可获得重建的解码图像。重建图像与原始图像相比，由于存在量化的影响，部分信息与原始图像有所不同，产生失真(Distortion)。对重建图像进行滤波操作，例如，去块效应滤波(deblocking)，SAO或者ALF等滤波器，可以有效的降低量化所产生的失真程度。由于这些经过滤波后的重建图像，将作为后续编码图像的参考，用于对将来的信号进行预测，所以上述的滤波操作也被称为环路滤波，及在编码环路内的滤波操作。
根据上述编码过程，可以看出，在解码端，对于每一个CU，解码器获得压缩码流后，先进行熵解码，获得各种模式信息及量化后的变换系数。各个系数经过反量化及反变换，得到残差信号。另一方面，根据已知的编码模式信息，可获得该CU对应的预测信号，两者相加之后，即可得到重建信号。最后，解码图像的重建值，需要经过环路滤波的操作，产生最终的输出信号。
(2)普通帧间预测模式和帧内块复制模式。
帧间预测或者帧内块复制模式的预测像素是通过在已重建的参考帧或者当前帧的已重建区域进行搜索或者在其亚像素的插值图像中搜索得到。对于色度分量，一般默认其与亮度分量具有类似的运动矢量或者块矢量，所以色度的运动矢量或者块矢量是基于亮度的运动矢量或者块矢量简单推导得到的。由于这个匹配搜索的过程不仅要考虑到图像的匹配度，还需要考虑到运动矢量或者块矢量的编码代价，所以单在匹配度这一维度不一定是最优的，也就是预测精度仍然存在一定的上升空间。
(3)单向运动信息和双向运动信息。
对于帧间编码模式，根据时域上参考帧相对当前帧的前后关系，可以分为前向参考帧和后向参考帧。运动信息一般包括参考帧索引和运动矢量(Motion Vector，MV)。如果一个运动信息含有1个运动信息，那么称之为单向运动信息；如果包含2个运动信息，则称之为双向运动信息。注意双向运动信息不是说存在前向运动信息和后向运动信息，也可能都是前向的。类似地，对于帧内块复制模式，因为搜索区域肯定在当前帧，所以帧内块复制模式的运动信息不会有参考帧索引。
(4)整像素和分像素。
图5是本申请实施例提供的整像素和分像素的示例。
如图5所示，黑色方块代表整像素(一帧所有整像素的分辨率等于视频分辨率)，然而物体的运动或者纹理的变化往往不一定是整像素的，所以相关的运动矢量也可以不是整像素的，可以指向图中灰色圆形的1/4分像素位置。
(5)运动矢量预测(Motion Vector Prediction，MVP)。
时域运动矢量预测(TMVP)：也就是运动矢量来自于参考帧。例如，可以选择时域参考帧上，与当前块(块内子块、块外子块)对应位置的编码块MV，(直接或者简单偏移后)作为当前块的运动矢量预测。
空域运动矢量预测(SMVP)：也就是运动矢量来自于当前帧(已编码/解码区域)。例如，可以选取当前块周围已编码区域的编码块的MV作为(直接或者简单偏移后)作为当前块的运动矢量预测。比如AVS3中，如图6所示，从块外子块F、G、C、A、B和D中搜索一个双向、后向、以及前向运动信息，加入运动矢量预测列表。
基于历史信息的运动矢量预测(History based Motion Vector Prediction，HMVP)：HMVP是基于历史信息的运动矢量预测。HMVP技术从先前编码块拷贝8个运动信息候选到FIFO中，FIFO以先进先出的方式保持不断更新。如果FIFO中的运动信息候选，与刚刚编码完成的运动信息相同，这个重复的候选首先会被移除，在FIFO的尾部会加入当前编码单元的运动信息。如果当前编码单元的运动信息与FIFO中任意候选的运动信息均不相同，就会去掉FIFO中第一个候选，再将最新的运动信息加到FIFO末尾，保证FIFO中永远保留8个最新的运动候选。
(6)运动搜索方法。
三步搜索方法：采用如图7所示的搜索图形进行搜索，每次搜索搜索步长变为一半，共搜索三次迭代搜索，每次迭代搜索选择SAD最小的搜索点作为下一次迭代搜索的起始点。
四步搜索方法：采用如图8中的(a)所示的搜索图形进行搜索，首先搜索图像为5x5，迭代搜索直至中心最优，则将搜索图像放缩到如图8中的(b)所示的搜索图形，再进行一次搜索得到最佳的搜索点。
菱形搜索方法：采用如图9中的(a)所示的搜索图形进行搜索，迭代搜索直至中心最优，然后采用如图9中的(b)所示的搜索图形继续进行搜索，再进行一次搜索得到最佳的搜索点。
六边形搜索方法：采用如图10中的(a)所示的搜索图形进行搜索，迭代搜索直至中心最优，然后采用如图10中的(b)所示的搜索图形继续进行搜索一次搜索，再用如图10中的(c)所示的搜索图形继续进行搜索一次搜索得到最佳的搜索点。
需要说明的是，本申请书中相关数据收集处理(例如用于进行编码的视频或图像的收集和处理)在应用时应该严格根据相关国家法律法规的要求，获取个人信息主体的知情同意或单独同意，并且在法律法规及个人信息主体的授权范围内，开展后续数据使用及处理行为。其中，数据收集可以通过收集日志的形式实现，例如可以通过获取目标存储器或目标处理器的日志，即可以获取相关数据。在进行日志收集时，收集粒度可以天为单位或以星期为单位等，本申请对此不作具体限定。
下面结合附图介绍本申请实施例提供的基于模板匹配的预测方法。该预测方法可以应用到具备相关视频编解码器或视频压缩的产品中。例如，本申请实施例提供的预测方法可以由编码端执行，例如图1所示的编码设备110或视频编码器112，再如图2所示的视频编码器200。再如，本申请实施例提供的预测方法可以由解码端执行，例如图1所示的解码设备120或视频解码器122，再如图4所示的视频解码器300。为便于说明，下面以预测装置为例对该预测方法进行说明。
图11是根据本申请实施例的基于模板匹配的预测方法400的示意性流程图。
如图11所示，该预测方法400可以包括如下内容中的至少部分内容：
S410，预测装置基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点。
示例性地，该VP列表包括但不限于运动矢量预测(Motion Vector Prediction，MVP)列表或块矢量预测(Block Vector Prediction，BVP)列表。该VP列表包括该MVP列表时，该第一VP为第一MVP，该第一矢量为MV。该VP列表包括BVP列表时，该第一VP为BVP，该第一矢量为BV。
S420，预测装置基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量。
示例性地，该当前块的模板可以由当前块周围的已重建像素组成，该当前块周围的已重建像素包括该当前块周围的以下至少一项位置上的已重建像素：左、上、左上、左下、右上。例如，图12给出了由当前块周围的已重建像素组成的8种模板的示例，当然，本申请并不限于图12给出的模板示例。
示例性地，所述当前块的参考区域与所述当前块的预测模式相关。例如，所述当前块的预测模式可以是帧间预测或帧内块复制模式。在所述当前块的预测模式为帧间预测模式的情况下，所述参考区域位于所述当前块的参考帧内；在所述当前块的预测模式为帧内块复制模式的情况下，所述参考区域在所述当前块所属的当前图像的已重建区域内。所述参考区域也可称为搜索区域。
示例性地，预测装置基于该第一起始搜索点，利用由该当前块的模板相对各个候选搜索点的模板的代价，在该当前块的参考区域内搜索该当前块的第二矢量。例如，预测装置基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板相对各个候选搜索点的模板的代价，在该当前块的参考区域内的至少一个候选搜索点内，搜索该当前块的第二矢量。
应当理解，本申请涉及的候选搜索点的模板可以理解为模板匹配过程中基于候选搜索点匹配的模板或基于候选搜索点搜索到的模板，本申请的对其具体描述方式不作限定。
S430，预测装置基于该第二矢量，确定该当前块的预测块。
示例性地，预测装置基于该第二矢量指向的第一预测块，确定该当前块的预测块。
本实施例中，基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量；基于该第二矢量，确定该当前块的预测块。相当于，该基于模板匹配的预测方法可以基于该第一起始搜索点，借助当前块周围已重建像素构建模板，在该当前块的参考区域内搜索该当前块的第二矢量，进而基于该第二矢量，确定该当前块的预测块。由于编码端和解码端都可以采用该基于模板匹配的预测方法确定该当前块的预测块，即编码端可以不需要传递矢量信息到解码端，可以节省运动信息传递时所占用的码率，进而能够提升码率。
此外，基于该第一起始搜索点搜索得到第二矢量，进而基于该第二矢量确定该当前块的预测块，相当于对该第一矢量进行了优化，能够获得较高质量的预测块，进而提升了该当前块的预测效果。
在一些实施例中，该第一VP通过第一标识指示，该第一标识通过码流携带或通过解码端推导确定；或者，该第一VP为该VP列表中模板代价最小的VP。
应当理解，本申请中，标识(例如第一标识或其他标识)通过解码端推导确定，旨在说明：编码端和解码端都可以通过推导的方式确定该标识，或者说，编码端不需要将标识写入码流，以及解码端不需要通过对码流的解码获取标识，本申请对其具体推导过程不作限定。标识通过解码端推导确定，也可以等同替换为：标识通过隐式指示确定，或标识通过隐式推导确定。标识通过码流携带，旨在说明：解码端将标识写入码流，解码端通过对码流进行解码得到标识。
需要说明的是，该第一VP可以是该VP列表中模板代价最小的VP，也可以是该VP列表中的其他VP，本申请对此不作具体限定。例如，在其他可替代实施例中，该第一VP可以是基于当前块的尺寸或其他信息在该VP列表中确定的VP。
需要说明的是，在其他可替代实施例中，本申请中对第二矢量的搜索也可以等同替换为对第二矢量指向的第一预测块的搜索，或者等同替换为对第二矢量指向的第一预测块的模板的搜索，本申请对此不作具体限定。例如，以替换为对第二矢量指向的第一预测块的搜索为例，预测装置可以基于当前块的VP列表中的第一VP所包括的第一矢量，确定第一起始搜索点；基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索第一预测块；基于该第一预测块矢量，确定该当前块的预测块。
在一些实施例中，该第一标识通过码流携带时，该第一标识的码字长度与该第一VP的模板代价正相关。
示例性地，预测装置作为编码端时，可以对VP列表中的VP进行模板代价评估，然后按照模板代价从小到大的顺序，对VP列表中的VP进行排序，使得模板代价最小的VP对应的码字长度最短。例如，假设VP列表中存在4个VP，则按照模板代价从小到大的顺序，该第一标识指示这该VP列表中的4个VP时，其对应的码字依次可以是1，01，001，000。
当然，在其他可替代实施例中，该第一标识的码字长度与该第一VP的模板代价也可以没有关系，例如，该第一标识的码字长度根据该VP列表中VP列表的数量确定，或者为固定长度。
在一些实施例中，该S410之前，该方法400还包括：
基于相邻空域VP、时域VP、非相邻空域VP、历史VP、平均VP、零矢量的VP中的至少一项，构建初始列表；对该初始列表中的初始VP执行去重复、矢量调整、模板代价评估、基于模板代价的排序中的至少一项，得到该VP列表。
示例性地，相邻空域VP指：基于当前块的空间相邻已解码块的矢量来预测当前块的矢量。由于视频序列中的连续性，相邻块的矢量往往与当前块的矢量相似。时域VP指：利用已解码的参考图像中相应位置块的矢量来预测当前块的矢量。这有助于捕捉跨多个帧的运动连贯性。非相邻空域VP指：与相邻空域VP类似，但考虑了更远或更非直接的相邻块。这些块可能位于当前块的同一行或列上，或者基于某种搜索模式来确定。历史VP指：使用已解码的块(可能位于不同的帧或不同的编码单元)的矢量来预测当前块的矢量。这种方法可以捕获时间跨度较大的运动模式或周期性运动。平均VP指：通过对多个已解码块(可能是相邻的或非相邻的)的矢量进行平均计算来得到预测的矢量。这种方法有助于平滑运动矢量的变化，并减少预测误差。零矢量的VP指：对于小运动或静止块，使用零矢量作为预测的矢量。这有助于减少编码小运动或静止块所需的比特数。
例如，以相邻空域MVP、时域MVP、非相邻空域MVP、历史MVP、平均MVP、零矢量的MVP为例：相邻空域MVP指：基于当前块的空间相邻已解码块的运动矢量来预测当前块的运动矢量。由于视频序列中的连续性，相邻块的运动矢量往往与当前块的运动矢量相似。时域MVP指：利用已解码的参考图像中相应位置块的运动矢量来预测当前块的运动矢量。这有助于捕捉跨多个帧的运动连贯性。非相邻空域MVP指：与相邻空域MVP类似，但考虑了更远或更非直接的相邻块。这些块可能位于当前块的同一行或列上，或者基于某种搜索模式来确定。历史MVP指：使用已解码的块(可能位于不同的帧或不同的编码单元)的运动矢量来预测当前块的运动矢量。这种方法可以捕获时间跨度较大的运动模式或周期性运动。平均MVP指：通过对多个已解码块(可能是相邻的或非相邻的)的运动矢量进行平均计算来得到预测的运动矢量。这种方法有助于平滑运动矢量的变化，并减少预测误差。零矢量的MVP指：对于小运动或静止块，使用零矢量作为预测的运动矢量。这有助于减少编码小运动或静止块所需的比特数。
应当理解，本申请中初始列表的构建过程可参考MVP列表的构建过程或BVP列表的构建过程。
以MVP列表的构建过程为例，可以利用帧间预测模式的合并(merge)/跳过(skip)/直接(direct)模式/提前运动矢量预测(Advanced motion vector prediction，AMVP)模式的MVP列表构建过程，构建该初始列表。例如，可以基于当前块周围的已编解码块(即空域MVP)、时域关联位置的已编解码块(即时域MVP)、历史MVP(History based Motion Vector Prediction，HMVP)、零矢量的MVP，构建该初始列表。再如，可以选择时域的MV构建两个单向的运动信息(分别指向前、后向参考帧列表)构成一个MVP，来构建该初始列表。
例如，类似于直接或跳过模式的MVP列表构建方式，构建一个长度为4的MVP列表，依次为：时域MVP、由相邻空域MVP构建的一个双向MVP、一个后向MVP和一个前向MVP。
示例性地，预测装置可以对该初始列表中的初始VP进行去重复处理，即删掉该初始列表中重复的初始VP。
示例性地，预测装置可以对该初始列表中的初始VP进行矢量调整。例如，预测装置可以对该初始列表中的初始VP所指向的位置进行调整。再如，预测装置可以对该初始列表中的初始VP的矢量的至少一个维度进行调整。
示例性地，预测装置可以对该初始列表中的初始VP进行模板代价评估。
示例性地，预测装置可以对该初始列表中的初始VP进行基于模板代价的排序。例如，预测装置可以对该初始列表中的初始VP进行模板代价评估，然后对该初始列表中的初始VP进行基于模板代价的排序。
需要说明的是，预测装置对该初始列表中的初始VP执行去重复、矢量调整、模板代价评估、基于模板代价的排序中的至少一项时，去重复、矢量调整、模板代价评估、基于模板代价的排序的顺序可以灵活调整。以预测装置对该初始列表中的初始VP执行去重复、矢量调整以及模板代价评估为例，其执行顺序可以是：去重复、模板代价评估、矢量调整，也可以是：模板代价评估、矢量调整、去重复等，还可以是：去重复、矢量调整、模板代价评估。
在一些实施例中，该S410包括：
将该第一矢量所指向的位置，确定为该第一起始搜索点；或
对该第一矢量的至少一个维度进行调整，得到该第一矢量的调整矢量，以及将该调整矢量所指向的位置，确定为该第一起始搜索点。
示例性地，所述至少一个维度对应所述第一矢量的至少一个数值。例如，预测装置可以对该第一矢量的至少一个数值进行调整，得到该第一矢量的调整矢量，以及将该调整矢量所指向的位置，确定为该第一起始搜索点。可选的，所述至少一个维度可以通过协议约定、通过码流携带或通过解码端推导确定。
在一些实施例中，对该第一矢量的至少一个维度进行调整，得到该第一矢量的第一调整矢量，包括以下至少一项：
基于该第一矢量的矢量残差VD，对该第一矢量的至少一个维度进行调整，得到该调整矢量；
基于该第一矢量确定移动位数，基于该移动位数对该第一矢量的至少一个维度进行右移计算，以及基于该移动位数对右移后的计算结果进行左移计算，得到该调整矢量；
将该第一矢量的至少一个维度，调整为指向距离该第一矢量所指向的位置最近的整像素位置，得到该调整矢量；
基于预设的偏移量，对该第一矢量的至少一个维度进行偏移，得到该调整矢量。
示例性地，该第一矢量为MV时，该第一矢量的VD为运动矢量残差(motion vector difference，MVD)。类似的，该第一矢量为BV时，该第一矢量的VD为块矢量残差(block vector difference，BVD)。例如，该第一矢量为AMVP模式的MV时，该第一矢量的VD为MVD，即预测装置可以基于(MV+MVD)进行模板匹配得到更好的MV(即第二矢量)。MVD是通过对码流解析得到的。再如，该第一矢量为IBC的MV时，该第一矢量的VD为BVD，即预测装置可以基于(BV+BVD)进行模板匹配得到更好的BV(即第二矢量)。BVD是通过对码流解析得到的。
示例性地，预测装置可以基于该第一矢量确定移动位数，基于该移动位数对该第一矢量的至少一个维度进行右移计算，以及基于该移动位数对右移后的计算结果进行左移计算，得到该调整矢量。本实施例中，预测装置可以直接将该第一矢量的分像素取值清零，使该第一矢量为整像素取值。可选的，预测装置可以将与该第一矢量的精度，将与该第一数量的精度对应的位数，确定为该移动位数。例如，以第一矢量为(MV_x，MV_y)为例，如果该第一矢量的精度是1/4像素，则预测装置可以按照以下方式对该第一矢量的每一个维度进行移位运算，得到该调整矢量：
(MV_x>>2)<<2，(MV_y>>2)<<2。
示例性地，预测装置可以将该第一矢量的至少一个维度，调整为指向距离该第一矢量所指向的位置最近的整像素位置，得到该调整矢量。换句话说，该第一矢量所指向的位置也可以称为原指向位置，即预测装置可以将该第一矢量的至少一个维度，调整为指向与原指向位置最近的整像素。当然，在其他可替代实施例中，预测装置可以将该第一矢量的至少一个维度，调整为指向距离该第一矢量所指向的位置最近的其他分像素位置，得到该调整矢量，本申请对此不作具体限定。
示例性地，预测装置可以基于预设的偏移量，对该第一矢量的至少一个维度进行偏移，得到该调整矢量。例如，以第一矢量为(MV_x，MV_y)为例，预测装置可以对MV_x和MV_y分别进行1/4，1/2或3/4像素的偏移，得到该调整矢量。
示例性地，预测装置可以基于该第一矢量确定移动位数，基于该移动位数对该第一矢量的第一维度进行右移计算，以及基于该移动位数对右移后的计算结果进行左移计算，以及预测装置可以基于预设的偏移量，对该第一矢量的第二维度进行偏移，得到该调整矢量。例如，在该第一维度为分像素取值并且该第二维度为整像素取值的情况下，预测装置可以基于该第一矢量确定移动位数，基于该移动位数对该第一矢量的第一维度进行右移计算，以及基于该移动位数对右移后的计算结果进行左移计算，以及预测装置可以基于预设的偏移量，对该第一矢量的第二维度进行偏移，得到该调整矢量。举例来说，对于1/4像素精度第一矢量(MV_x，MV_y)，如果一个维度是整像素的值，进行+2或-2调整到1/2像素位置；如果另一个维度是分像素，则进行先右移2位，再左移2位的操作，得到调整矢量。
示例性地，预测装置可以将该第一矢量的至少一个维度，调整为指向整像素位置，然后调整为指向分像素位置。例如，预测装置可以基于该第一矢量确定移动位数，基于该移动位数对该第一矢量的至少一个维度进行右移计算，以及基于该移动位数对右移后的计算结果进行左移计算，然后，可以基于预设的偏移量，对左移后的矢量的至少一个维度进行偏移，得到该调整矢量。再如，预测装置可以将该第一矢量的至少一个维度，调整为指向距离该第一矢量所指向的位置最近的整像素位置，然后，可以基于预设的偏移量，对指向距离该第一矢量所指向的位置最近的整像素位置的矢量的至少一个维度进行偏移，得到该调整矢量。
需要说明的是，本申请中，对第一矢量的维度进行调整，可以理解为对第一矢量在维度上的取值进行调整。当然，也可以理解为对具体维度进行调整。此外，在其他可替代实施例中，预测装置也可以基于预设的规则，对该第一矢量的至少一个维度进行偏移，得到该调整矢量，本申请对此不作具体限定。
在一些实施例中，该S420包括：
基于该第一起始搜索点，搜索至少一个候选搜索点；
将该第一矢量调整为指向该至少一个候选搜索点中模板代价最小的第一最佳搜索点，得到该第二矢量。
示例性地，该第一候选搜索点为该第一最佳搜索点时，预测装置可以直接将该第一矢量确定为该第二矢量；该第一候选搜索点不为该第一最佳搜索点时，预测装置将该第一矢量调整为指向该第一最佳搜索点，得到该第二矢量。
在一些实施例中，基于该第一起始搜索点，搜索至少一个候选搜索点，包括以下至少一项：
通过遍历搜索候选搜索点的方式，得到该至少一个候选搜索点；
基于该第一起始搜索点和该当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，直至满足预设条件的情况下，将该当前迭代过程中得到的模板代价最小的候选搜索点，确定为该第一最佳搜索点；
其中，该预设条件包括以下至少一项：
该当前迭代过程中得到的模板代价最小的候选搜索点为该当前迭代过程使用的起始搜索点；
已迭代次数大于或等于最大迭代次数；
已搜索的点大于或等于最大搜索点数。
示例性地，已搜索的点大于或等于最大搜索点数，可以理解为：已搜素的点的数量大于或等于最大搜索点数，例如，已搜索的并且经过去重复的点的数量大于或等于最大搜索点数，再如，已搜索的并且包括重复的点的数量大于或等于最大搜索点数。
示例性地，预测装置可以通过遍历该参考区域内的所有候选搜索点的方式，得到该至少一个候选搜索点。
示例性地，预测装置通过遍历搜索候选搜索点的方式，得到该至少一个候选搜索点搜索方法，相当于，预测装置可以是遍历参考区域中所有可能的候选搜索点，来寻找模板代价最小的第一最佳搜索点。
示例性地，预测装置可以基于该第一起始搜索点，基于该第一搜索形状，对该参考区域内模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，直至满足预设条件的情况下，将该当前迭代过程中得到的模板代价最小的候选搜索点，确定为该第一最佳搜索点。值得注意的是，预测装置进行迭代搜索时，该至少一个候选搜索点包括每一次迭代过程中得到的候选搜索点。其中，不同的迭代过程中得到的候选搜索点可以不重叠，也可以部分重叠，本申请对此不作具体限定。
示例性地，预测装置通过遍历搜索候选搜索点的方式，得到该至少一个候选搜索点后，可以对该至少一个候选搜索点进行模板代价评估，并将该第一矢量调整为指向该至少一个候选搜索点中模板代价最小的第一最佳搜索点，得到该第二矢量。
示例性地，预测装置对模板代价最小的候选搜索点进行迭代搜索得到当前迭代过程中得到的候选搜索点后，可以对当前迭代过程中得到的候选搜索点进行去重复，例如去掉之前已经排除过的候选搜索点，然后对去重复后的候选搜索点进行模板代价评估，并将该第一矢量调整为指向去重复后的候选搜索点中模板代价最小的第一最佳搜索点，得到该第二矢量。
示例性地，预测装置进行模板代价评估时，如图13所示，每一次搜索获取候选搜索点的模板并计算其和当前块的模板之间的差异，差异采用绝对差值和(sum of absolute difference，SAD)、平方差值和(sum of squared difference，SSD)、平均绝对差值(mean of absolute difference，MAD)、平均平方差值(mean of squared difference，MSD)等评判。目标是尽量快速的在参考区域内找到模板与当前块的模板之间的差异最小的候选搜索点，并基于其确定当前块的预测块。
应当理解，本实施例中，预测装置进行迭代搜索的截至条件包括以下至少一项：
该当前迭代过程中得到的模板代价最小的候选搜索点为该当前迭代过程使用的起始搜索点；
已迭代次数大于或等于最大迭代次数；
已搜索的点大于或等于最大搜索点数。
上述截至条件可以单独使用，也可以组合使用。
例如，预测装置可以基于该第一起始搜索点和该当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，直至该当前迭代过程中得到的模板代价最小的候选搜索点为该当前迭代过程使用的起始搜索点的情况下，将该当前迭代过程中得到的模板代价最小的候选搜索点，确定为该第一最佳搜索点。
再如，预测装置可以基于该第一起始搜索点和该当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，直至满足以下条件中的任意一项的情况下，将该当前迭代过程中得到的模板代价最小的候选搜索点，确定为该第一最佳搜索点。
该当前迭代过程中得到的模板代价最小的候选搜索点为该当前迭代过程使用的起始搜索点；
已迭代次数大于或等于最大迭代次数；
已搜索的点大于或等于最大搜索点数。
在一些实施例中，基于该第一起始搜索点和该当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，包括：
基于该第一起始搜索点和该当前块使用的搜索方法，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点；其中，该搜索方法包括至少一个搜索形状和该至少一个搜索形状中的每一个搜索形状对应的迭代次数，该至少一个搜索形状包括该第一搜索形状。
示例性地，预测装置对模板代价最小的候选搜索点进行迭代搜索，相当于，在搜索速度和寻找最佳模板预测块中间取一个平衡。模板匹配的算法与编码端运动搜索的方法是类似的，比如常见的方法有三步搜索方法、四步搜索方法、菱形搜索方法、六边形搜索方法、非对称十字形多层次六边形格点搜索方法。
图14是本申请实施例提供的搜索方法包括的搜索形状的示例。
如图14所示，搜索方法包括的搜索形状可以如(a)～(g)中的任意形状，每个形状中的五角星即为一次迭代过程使用的起始搜索点，五角星和所有的正方形为一次迭代过程中需要搜索的候选搜索点。通过对搜索到的候选搜索点的模板代价进行对比，得到模板代价最小的候选搜索点，称为一次迭代过程。本申请中的当前迭代过程可以理解为当前次(或当前轮次)的迭代过程或当前执行的一次迭代过程。
预测装置对模板最小的候选搜索点进行迭代搜索时，可通过多次的迭代过程，对模板最小的候选搜索点进行搜索。
具体地，预测装置在得到当前迭代过程中模板代价最小的候选搜索点后，在满足预设条件的情况下，将该当前迭代过程中得到的模板代价最小的候选搜索点，确定为该第一最佳搜索点；在不满足预设条件的情况下，会重新执行一次迭代过程，即将已得到的模板代价最小的候选搜索点(即，最近一次迭代过程中得到的模板代价最小的候选搜索点)作为当前迭代过程使用的起始搜索点(即五角星位置)，对模板代价最小的候选搜索点进行搜索，得到当前迭代过程中模板代价最小的候选搜索点，以此类推，直至在满足预设条件的情况下，将当前迭代过程中得到的模板代价最小的候选搜索点，确定为最终的最佳搜索点(即第一最佳搜索点)。该预设条件可以是：当前迭代过程使用的起始搜索点(即五角星位置)为模板代价最小的候选搜索点；或者，该预设条件也可以使用(或结合)最大迭代次数/最大搜索点数为N(正整数)。
应当理解，图中搜索点的位置间距仅为示意，可以按照等比例进行放缩。
在一种实现方式中，整个搜索过程中的不同的迭代过程可以使用相同的搜索形状。
需要注意的是，虽然使用同一个搜索形状，但是候选搜索点的间距(也可以称为搜索步长)可以放缩，例如可以随着迭代次数的增加，经过至少1次迭代搜索后，候选搜索点的间距会按比例缩小。例如，如图7所示的三步搜索方法。
在另一种实现方式中，不同的搜索形状可以像编码端运动估计时使用的搜索方法那样进行组合形成整个的搜索方法；列举3个实际搜索的例子：
示例1：先采用(a)所示的大菱形进行迭代搜索，限制最大迭代次数为30次，等搜索终止后，再用(e)所示的小菱形搜索一次。
示例2：先采用的(d)所示的六边形搜索，限制最大迭代次数为30次，再用(e)所示的小菱形搜索1次，再用(g)所示的搜索1次。
示例3：先采用的(d)所示的六边形搜索，限制最大迭代次数为40次，再用(f)所示的搜索1次。
应当理解，本实施例中，该当前块使用的搜索方法可以称为基于模板匹配的搜索方法，该基于模板匹配的搜索方法与编码端的运动估计时使用的搜索方法可以相同或不同，本申请对此不做具体限定。
在一些实施例中，基于该第一起始搜索点和该当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，包括：
基于该第一起始搜索点和该第一搜索形状，对模板代价最小的候选搜索点进行第一迭代过程，得到该第一迭代过程中得到的候选搜索点，以及基于该第一迭代过程中得到的候选搜索点中模板代价最小的候选搜索点，对模板代价最小的候选搜索点进行第二迭代过程，得到该第二迭代过程中得到的候选搜索点；
在该第一迭代过程中得到的第一候选搜索点与该第二迭代过程中得到的第二候选搜索点重叠的情况下，将该第一候选搜索点的模板代价，确定为该第二候选搜索点的模板代价；或者
在该第一候选搜索点和该第二候选搜索点重叠、并且该第二候选搜索点不是该第二迭代过程使用的起始搜索点的情况下，忽略该第二候选搜索点，或基于该第二迭代过程中得到的除该第二候选搜索点之外的候选搜索点，确定该第二迭代过程中模板代价最小的候选搜索点。
换言之，在不同的迭代搜索过程中，如果存在重合的候选搜索点，则可以加载之前的模板代价，不用重新计算以节省复杂度。或者说，如果重合的候选搜索点不是当前迭代过程使用的搜索起点(例如，如图14所示的搜索形状的五角星位置)，那可以直接放弃这个重合的候选搜索点的代价计算(因为之前没有选中不可能是最佳的了)，以节省搜索复杂度。
在一些实施例中，该至少一个候选搜索点中任意候选搜索点的第一偏移量的绝对值小于或等于第三预设阈值，或该任意候选搜索点的第二偏移量的绝对值小于或等于第四预设阈值，或该任意候选搜索点的第一偏移量的绝对值和第二偏移量的绝对值和小于或等于第五预设阈值；其中，该第一偏移量为该任意候选搜索点在第一维度上相对该第一起始搜索点的偏移量，该第二偏移量为该任意候选搜索点在第二维度上相对该第一起始搜索点的偏移量。
示例性地，该第一维度为水平方向，该第二维度为垂直方向。
示例性地，以第一起始搜索点的位置为(MV0_x，MV0_y)为例，该任意候选搜索点在水平方向上和竖直方向上，相对该第一起始搜索点的偏移的绝对值(|MV_x-MV0_x|，|MV_y-MV0_y|)分别不大于M(正整数)和N(正整数)，比如，M和N分别为8个整像素。
示例性地，以第一起始搜索点的位置为(MV0_x，MV0_y)为例，该任意候选搜索点在水平方向上和竖直方向上，相对该第一起始搜索点的偏移的绝对值之和(|MV_x-MV0_x|+|MV_y-MV0_y|)不超过M(正整数)，比如，M设置为10个整像素。
应当理解，本申请在搜索至少一个候选搜索点时，可以对搜索范围不限定，也可以进行限定，对搜索范围进行限定时，约束搜索过程只能在限定的搜索范围内展开，能够降低复杂度和硬件实现代价。
在一些实施例中，得到该第二矢量之前，该方法400还包括：
基于该当前块的多个模板区域，确定该多个模板区域的代价权重；基于该多个模板区域的代价权重，对该多个模板区域相对该至少一个候选搜索点中任意候选搜索点的模板的代价进行加权，得到该任意候选搜索点的模板代价；其中，该多个模板区域中第一模板区域的代价权重满足以下至少一项：
该第一模板区域的代价权重与第一尺寸正相关，该第一尺寸为该当前块的与该第一模板区域相邻的边长的尺寸；
该第一模板区域的代价权重与第一距离负相关，该第一距离为该第一模板区域与该当前块之间的距离。
示例性地，预测装置基于该当前块的多个模板区域，确定该多个模板区域的代价权重；基于该多个模板区域的代价权重，对该多个模板区域相对该至少一个候选搜索点中任意候选搜索点的模板的代价进行加权，得到该任意候选搜索点的模板代价；其中，该第一模板区域的代价权重与该第一尺寸正相关。
图15是本申请实施例提供的多个模板区域的示例。
如图15所示，可以根据当前块的尺寸对当前块的多个模板区域进行有区别的加权。靠近较长边的模板区域的代价权重更大。比如，当前块的宽度大于高度时，对于如图15中的(a)所示的两个模板区域，W1＞W2，或对于如图15中的(b)所示的三个模板区域，W1＞W2＝W3；当前块的高度大于宽度时，对于如图15中的(a)所示的两个模板区域，W2＞W1，或对于如图15中的(b)所示的三个模板区域，W2＞W1＝W3；当宽度等于高度时，对于如图15中的(a)所示的两个模板区域，W1＝W2，或对于如图15中的(b)所示的三个模板区域，W1＝W2＝W3。
示例性地，预测装置基于该当前块的多个模板区域，确定该多个模板区域的代价权重；基于该多个模板区域的代价权重，对该多个模板区域相对该至少一个候选搜索点中任意候选搜索点的模板的代价进行加权，得到该任意候选搜索点的模板代价；其中，该第一模板区域的代价权重与该第一距离负相关。
图16是本申请实施例提供的多个模板区域的另一示例。
如图16所示，可以根据距离当前块的距离对当前块的模板区域进行有区别的加权。比如，如图16中的(a)或16中的(b)所示，假设根据距离当前块的距离将当前块的模板划分为4个模板区域(也可以称为4类模板区域或4个子区)，可以分开计算不同模板区域的代价，然后进行加权，距离当前块距离越近，代价权重越大。比如，可以按照距离由大到小的顺序设置代价权重为(1，2，3，4)或(1，2，2，3)。
当然，上文涉及的第一尺寸和第一距离也可以结合起来，当宽高相等时，相当于，仅考虑第一距离；当宽高不等时，在考虑第一距离的基础上，靠近长边的模板区域设置更大的代价权重。
在一些实施例中，该S430包括以下任一项：
将该第二矢量指向的第一预测块，确定为该当前块的预测块；
对该第二矢量指向的第一预测块和至少一个矢量指向的预测块进行融合，得到该当前块的预测块；其中，该第一预测块的融合权重和该至少一个矢量中任意矢量指向的预测块的融合权重相等，或该第一预测块的融合权重和该任意矢量指向的预测块的融合权重与模板代价负相关；
其中，该至少一个矢量包括以下至少一项：
基于该第一起始搜索点搜索的除该第二矢量之外的矢量；
该第一VP所包括的除该第一矢量之外的第三矢量；
基于该第三矢量确定第二起始搜索点，以及基于该第二起始搜索点，利用该当前块的模板在该搜索区域内搜索得到的第四矢量；
基于该第二起始搜索点搜索的除该第四矢量之外的矢量；
基于该VP列表中除该第一VP之外的第二VP所包括的第五矢量；
基于该第五矢量确定第三起始搜索点，以及基于该第三起始搜索点，利用该当前块的模板在该搜索区域内搜索得到的第六矢量。
示例性地，基于该第一起始搜索点搜索的除该第二矢量之外的矢量，可以是上文涉及的至少一个候选搜索点中按照模板代价由小到大的顺序选择的除第一最佳搜索点之外的一个或多个次佳候选搜索点，以及基于该一个或多个次佳候选搜索点确定的一个或多个矢量。
示例性地，基于该第二起始搜索点搜索的除该第四矢量之外的矢量的确定方法，和基于该第一起始搜索点搜索的除该第二矢量之外的矢量的确定方法，可以相同或者不同，本申请对此不作具体限定。
示例性地，预测装置可以将该第二矢量指向的第一预测块，确定为该当前块的预测块。相当于，预测装置可以通过对第一矢量进行模板匹配优化得到第二矢量，并将该第二矢量指向的第一预测块，确定为该当前块的预测块。例如，在该第一VP仅包括该第一矢量的情况下，预测装置可以将该第二矢量指向的第一预测块，确定为该当前块的预测块。
示例性地，预测装置可以对该第二矢量指向的第一预测块和以下预测块中的至少一项进行融合，得到该当前块的预测块：基于该第一起始搜索点搜索的除该第二矢量之外的矢量指向的预测块、该第三矢量指向的预测块、该第四矢量指向的预测块、基于该第二起始搜索点搜索的除该第四矢量之外的矢量指向的预测块、该第五矢量指向的预测块、该第六矢量指向的预测块。
以对该第一预测块和该第四矢量指向的预测块进行融合为例，预测装置可以对该第一预测块和该第四矢量指向的预测块进行融合，得到该当前块的预测块。相当于，预测装置可以先通过对第一矢量进行模板匹配优化得到第二矢量，并通过对第三矢量进行模板匹配优化得到第四矢量，然后对该第一预测块和该第四矢量指向的预测块进行融合，得到该当前块的预测块。例如，在该第一VP包括该第三矢量的情况下，预测装置可以对该第一预测块和该第四矢量指向的预测块进行融合，得到该当前块的预测块。其中，该第一预测块的融合权重和该第四矢量指向的预测块的融合权重相等，或该第一预测块的融合权重和该第四矢量指向的预测块的融合权重与模板代价负相关。例如，该第一预测块的融合权重和第一预测块的模板的代价负相关，该第四矢量指向的预测块的融合权重与该第四矢量指向的预测块的模板的代价负相关。
示例性地，该至少一个矢量通过一个标识指示，这个标识通过码流携带或通过解码端推导确定。
需要说明的是，本申请对该至少一个矢量的确定方式不作具体限定。例如，预测装置可以基于当前块的尺寸、该第一VP包括的矢量的数量或其他信息确定该至少一个矢量。
在一些实施例中，该方法400还包括以下至少一项：
基于该第二起始搜索点，搜索模板代价最小的第二最佳搜索点，基于该第二最佳搜索点和第一最佳搜索点中模板代价最大的最佳搜索点，搜索模板与第一模板融合后模板代价最小的第三最佳搜索点，以及将该模板代价最大的最佳搜索点，更新为该第三最佳搜索点；
基于该第二起始搜索点，搜索模板与第一最佳搜索点的模板融合后模板代价最小的候选搜索点，得到该第二最佳搜索点；
其中，该第一模板为该第一最佳搜索点和该第二最佳搜索点中模板代价最小的最佳搜索点的模板，该第一最佳搜索点为基于该第一起始搜索点搜索的模板代价最小的候选搜索点。
示例性地，与第一模板融合后模板代价指搜索的模板与该第一模板进行融合后的融合模板与当前块的模板之间的差异度。类似的，与第一最佳搜索点的模板融合后模板代价指搜索的模板与该第一最佳搜索点的模板进行融合后的融合模板与当前块的模板之间的差异度。
以该第一VP为MVP，该第一VP包括双向运动矢量(即两个运动矢量)为例，预测装置可以分别对两个运动矢量采用模板匹配的方式进行优化，然后基于得到的两个预测块，采用双向运动矢量预测的方式进行融合，一般就是加权处理。具体地，先对两个运动矢量，采用模板匹配的方式进行优化；然后对比两个运动矢量的模板代价，如果两个模板代价不相等，则需要进行耦合搜索。则固定代价较小的优化好的运动矢量(MV1)不变(即模板P1不变)，再采用模板匹配的方式基于代价较大的优化好的运动矢量(MV2)继续搜索最佳的匹配模板(P2’)，但是这个时候不再是根据模板和匹配模板的差异度作为代价，而是计算P1和P2’融合后的融合模板和当前块的模板之间的差异度作为代价，至于搜索方法则和前述的搜索方法一致或者类似，最终得到耦合搜索优化后的双向运动矢量(MV1，MV2’)。
在一些实施例中，该S410包括：
在满足以下至少一项的情况下，基于该第一矢量，确定该第一起始搜索点：
第二标识指示利用该当前块的模板搜索该当前块的矢量，该第二标识通过码流携带或通过解码端推导确定；
通过码流携带的第三标识指示该当前块所属的当前图像序列、当前图像或当前条带允许利用该当前块的模板搜索该当前块的矢量；
该当前块的预测模式是帧间预测模式或帧内块复制预测模式；
当前预测模式是帧间预测模式，且该当前块的参考帧列表包括前向参考帧和后向参考帧。
示例性地，在当前预测模式是帧间预测模式，且该当前块的参考帧列表包括前向参考帧和后向参考帧的情况下，预测装置基于该第一矢量，确定该第一起始搜索点；在当前预测模式是帧间预测模式，且该当前块的参考帧列表仅包括前向参考帧的情况下，预测装置不触发基于该第一矢量确定该第一起始搜索点的过程。或者说，在当前预测模式是帧间预测模式，且该当前块的参考帧列表包括前向参考帧和后向参考帧的情况下，预测装置基于模板匹配对该第一矢量进行优化；在当前预测模式是帧间预测模式，且该当前块的参考帧列表仅包括前向参考帧的情况下，预测装置禁止对该第一矢量进行优化。
示例性地，该第三标识通过码流携带或通过解码端推导确定。
在一些实施例中，该S420包括：
使用码流中携带的参数信息，在该参考区域内搜索该第二矢量；
其中，该参数信息包括以下至少一项：
迭代搜索使用的最大迭代次数、迭代搜索使用的最大搜索点数、迭代搜索使用的搜索方法、模板代价的计算方式。
以上结合附图详细描述了本申请的优选实施方式，但是，本申请并不限于上文涉及的实施方式中的具体细节，在本申请的技术构思范围内，可以对本申请的技术方案进行多种简单变型，这些简单变型均属于本申请的保护范围。例如，在上文涉及的具体实施方式中所描述的各个具体技术特征，在不矛盾的情况下，可以通过任何合适的方式进行组合，为了避免不必要的重复，本申请对各种可能的组合方式不再另行说明。又例如，本申请的各种不同的实施方式之间也可以进行任意组合，只要其不违背本申请的思想，其同样应当视为本申请所公开的内容。
还应理解，在本申请的各种方法实施例中，上文涉及的各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
上文对本申请实施例提供的方法进行了说明，下面对本申请实施例提供的装置进行说明。
图17是根据本申请实施例提供的基于模板匹配的预测装置500的示意性框图。
如图17所示，该预测装置500包括：
第一确定单元510，用于基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；
搜索单元520，用于基于该第一起始搜索点，利用由该当前块周围的已重建像素形成的模板，在该当前块的参考区域内搜索该当前块的第二矢量；
第二确定单元530，用于基于该第二矢量，确定该当前块的预测块。
在一些实施例中，该第一VP通过第一标识指示，该第一标识通过码流携带或通过解码端推导确定；或者，该第一VP为该VP列表中模板代价最小的VP。
在一些实施例中，该第一标识通过码流携带时，该第一标识的码字长度与该第一VP的模板代价正相关。
在一些实施例中，该第一确定单元510确定第一起始搜索点之前，还用于：
基于相邻空域VP、时域VP、非相邻空域VP、历史VP、平均VP、零矢量的VP中的至少一项，构建初始列表；
对该初始列表中的初始VP执行去重复、矢量调整、模板代价评估、基于模板代价的排序中的至少一项，得到该VP列表。
在一些实施例中，该第一确定单元510具体用于：
将该第一矢量所指向的位置，确定为该第一起始搜索点；或
对该第一矢量的至少一个维度进行调整，得到该第一矢量的调整矢量，以及将该调整矢量所指向的位置，确定为该第一起始搜索点。
在一些实施例中，该第一确定单元510具体用于执行以下至少一项：
基于该第一矢量的矢量残差VD，对该第一矢量的至少一个维度进行调整，得到该调整矢量；
基于该第一矢量确定移动位数，基于该移动位数对该第一矢量的至少一个维度进行右移计算，以及基于该移动位数对右移后的计算结果进行左移计算，得到该调整矢量；
将该第一矢量的至少一个维度，调整为指向距离该第一矢量所指向的位置最近的整像素位置，得到该调整矢量；
基于预设的偏移量，对该第一矢量的至少一个维度进行偏移，得到该调整矢量。
在一些实施例中，该搜索单元520具体用于：
基于该第一起始搜索点，搜索至少一个候选搜索点；
将该第一矢量调整为指向该至少一个候选搜索点中模板代价最小的第一最佳搜索点，得到该第二矢量。
在一些实施例中，该搜索单元520具体用于执行以下至少一项：
通过遍历搜索候选搜索点的方式，得到该至少一个候选搜索点；
基于该第一起始搜索点和该当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，直至满足预设条件的情况下，将该当前迭代过程中得到的模板代价最小的候选搜索点，确定为该第一最佳搜索点；
其中，该预设条件包括以下至少一项：
该当前迭代过程中得到的模板代价最小的候选搜索点为该当前迭代过程使用的起始搜索点；
已迭代次数大于或等于最大迭代次数；
已搜索的点大于或等于最大搜索点数。
在一些实施例中，该搜索单元520具体用于：
基于该第一起始搜索点和该当前块使用的搜索方法，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点；
其中，该搜索方法包括至少一个搜索形状和该至少一个搜索形状中的每一个搜索形状对应的迭代次数，该至少一个搜索形状包括该第一搜索形状。
在一些实施例中，该搜索单元520具体用于：
基于该第一搜索形状，对模板代价最小的候选搜索点进行第一迭代过程，得到该第一迭代过程中得到的候选搜索点，以及基于该第一迭代过程中得到的候选搜索点中模板代价最小的候选搜索点，对模板代价最小的候选搜索点进行第二迭代过程，得到该第二迭代过程中得到的候选搜索点；
在该第一迭代过程中得到的第一候选搜索点与该第二迭代过程中得到的第二候选搜索点重叠的情况下，将该第一候选搜索点的模板代价，确定为该第二候选搜索点的模板代价；或者
在该第一候选搜索点和该第二候选搜索点重叠、并且该第二候选搜索点不是该第二迭代过程使用的起始搜索点的情况下，忽略该第二候选搜索点，或基于该第二迭代过程中得到的除该第二候选搜索点之外的候选搜索点，确定该第二迭代过程中模板代价最小的候选搜索点。
在一些实施例中，该至少一个候选搜索点中任意候选搜索点的第一偏移量的绝对值小于或等于第三预设阈值，或该任意候选搜索点的第二偏移量的绝对值小于或等于第四预设阈值，或该任意候选搜索点的第一偏移量的绝对值和第二偏移量的绝对值和小于或等于第五预设阈值；
其中，该第一偏移量为该任意候选搜索点在第一维度上相对该第一起始搜索点的偏移量，该第二偏移量为该任意候选搜索点在第二维度上相对该第一起始搜索点的偏移量。
在一些实施例中，该搜索单元520得到该第二矢量之前，还用于：
基于该当前块的多个模板区域，确定该多个模板区域的代价权重；
基于该多个模板区域的代价权重，对该多个模板区域相对该至少一个候选搜索点中任意候选搜索点的模板的代价进行加权，得到该任意候选搜索点的模板代价；
其中，该多个模板区域中第一模板区域的代价权重满足以下至少一项：
该第一模板区域的代价权重与第一尺寸正相关，该第一尺寸为该当前块的与该第一模板区域相邻的边长的尺寸；
该第一模板区域的代价权重与第一距离负相关，该第一距离为该第一模板区域与该当前块之间的距离。
在一些实施例中，该第二确定单元530具体用于执行以下任一项：
将该第二矢量指向的第一预测块，确定为该当前块的预测块；
对该第二矢量指向的第一预测块和至少一个矢量指向的预测块进行融合，得到该当前块的预测块；其中，该第一预测块的融合权重和该至少一个矢量中任意矢量指向的预测块的融合权重相等，或该第一预测块的融合权重和该任意矢量指向的预测块的融合权重与模板代价负相关；
其中，该至少一个矢量包括以下至少一项：
基于该第一起始搜索点搜索的除该第二矢量之外的矢量；
该第一VP所包括的除该第一矢量之外的第三矢量；
基于该第三矢量确定第二起始搜索点，以及基于该第二起始搜索点，利用该当前块的模板在该搜索区域内搜索得到的第四矢量；
基于该第二起始搜索点搜索的除该第四矢量之外的矢量；
基于该VP列表中除该第一VP之外的第二VP所包括的第五矢量；
基于该第五矢量确定第三起始搜索点，以及基于该第三起始搜索点，利用该当前块的模板在该搜索区域内搜索得到的第六矢量。
在一些实施例中，该搜索单元520还用于执行以下至少一项：
基于该第二起始搜索点，搜索模板代价最小的第二最佳搜索点，基于该第二最佳搜索点和第一最佳搜索点中模板代价最大的最佳搜索点，搜索模板与第一模板融合后模板代价最小的第三最佳搜索点，以及将该模板代价最大的最佳搜索点，更新为该第三最佳搜索点；
基于该第二起始搜索点，搜索模板与第一最佳搜索点的模板融合后模板代价最小的候选搜索点，得到该第二最佳搜索点；
其中，该第一模板为该第一最佳搜索点和该第二最佳搜索点中模板代价最小的最佳搜索点的模板，该第一最佳搜索点为基于该第一起始搜索点搜索的模板代价最小的候选搜索点。
在一些实施例中，该第一确定单元510具体用于：
在满足以下至少一项的情况下，基于该第一矢量，确定该第一起始搜索点：
第二标识指示利用该当前块的模板搜索该当前块的矢量，该第二标识通过码流携带或通过解码端推导确定；
通过码流携带的第三标识指示该当前块所属的当前图像序列、当前图像或当前条带允许利用该当前块的模板搜索该当前块的矢量；
该当前块的预测模式是帧间预测模式或帧内块复制预测模式；
当前预测模式是帧间预测模式，且该当前块的参考帧列表包括前向参考帧和后向参考帧。
在一些实施例中，该搜索单元520具体用于：
使用码流中携带的参数信息，在该参考区域内搜索该第二矢量；
其中，该参数信息包括以下至少一项：
迭代搜索使用的最大迭代次数、迭代搜索使用的最大搜索点数、迭代搜索使用的搜索方法、模板代价的计算方式。
应理解，本申请实施例提供的预测装置500可对应于本申请方法实施例中的执行主体，并且预测装置500中的各个单元分别为了实现图11所示的预测方法400的相应流程，为了简洁，在此不再赘述。
本申请实施例提供的预测装置能够实现图11的方法实施例中的各个过程，并达到相同的技术效果，为避免重复，这里不再赘述。
还应理解，本申请实施例涉及的预测装置500中的各个单元是基于逻辑功能划分的，在实际应用中，一个单元的功能也可以由多个单元来实现，或者多个单元的功能由一个单元实现，甚至，这些功能也可以由一个或多个其它单元协助实现。例如，预测装置500中的部分或全部合并为一个或若干个另外的单元。再如，预测装置500中的某个(些)单元还可以再拆分为功能上更小的多个单元来构成，这可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。再如，预测装置500也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。
还应理解，本申请实施例涉及的术语“模块”或“单元”是指有预定功能的计算机程序或计算机程序的一部分，并与其他相关部分一起工作以实现预定目标，并且可以通过使用软件、硬件(如处理电路或存储器)或其组合来全部或部分实现。同样的，一个处理器(或多个处理器或存储器)可以用来实现一个或多个模块或单元。此外，每个模块或单元都可以是包含该模块或单元功能的整体模块或单元的一部分。
根据本申请的另一个实施例，可以通过在包括例如中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的通用计算机的通用计算设备上运行能够执行相应方法所涉及的各步骤的计算机程序(包括程序代码)，来构造本申请实施例涉及的预测装置500，以及来实现本申请实施例的方法。计算机程序可以记载于例如计算机可读存储介质上，并通过计算机可读存储介质装载于电子设备中，计算机程序用来实现本申请实施例的相应方法。换言之，上文涉及的单元可以通过硬件形式实现，也可以通过软件形式的指令实现，还可以通过软硬件结合的形式实现。具体地，本申请实施例中的方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路和/或软件形式的指令完成，结合本申请实施例公开的方法的步骤可以直接体现为利用硬件译码处理器执行完成，或者利用译码处理器中的硬件及软件组合执行完成。可选地，软件可以位于随机存储器，闪存、只读存储器、可编程只读存储器、电可擦写可编程存储器、寄存器等本领域的成熟的存储介质中。存储器中的软件可通过处理器运行，以完成上文涉及的方法实施例中的步骤。
图18是本申请提供的电子设备600的示意结构图。
如图18所示，该电子设备600至少包括处理器610以及计算机可读存储介质620。其中，处理器610以及计算机可读存储介质620可通过总线或者其它方式连接。计算机可读存储介质620用于存储计算机程序621，计算机程序621包括计算机指令，处理器610用于执行计算机可读存储介质620存储的计算机指令。处理器610是电子设备600的计算核心以及控制核心，其适于实现一条或多条计算机指令，具体适于加载并执行一条或多条计算机指令从而实现相应方法流程或相应功能。
处理器610也可称为中央处理器(Central Processing Unit，CPU)。处理器610可以包括但不限于：通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件、分立硬件组件等等。
计算机可读存储介质620可以是高速RAM存储器，也可以是非不稳定的存储器(Non-VolatileMemory)，例如至少一个磁盘存储器；可选的，还可以是至少一个位于远离前述处理器610的计算机可读存储介质。具体而言，计算机可读存储介质620包括但不限于：易失性存储器和/或非易失性存储器。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DR RAM)。
如图18所示，该电子设备600还可以包括收发器630。
其中，处理器610可以控制该收发器630与其他设备进行通信，具体地，可以向其他设备发送信息或数据，或接收其他设备发送的信息或数据。收发器630可以包括发射机和接收机。收发器630还可以进一步包括天线，天线的数量可以为一个或多个。
作为一种实现方式，该电子设备600可以是本申请实施例涉及的预测装置500；该计算机可读存储介质620中存储有计算机指令；由处理器610加载并执行计算机可读存储介质620中存放的计算机指令，以实现本申请提供的预测方法400中的相应步骤；换言之，计算机可读存储介质620中的计算机指令由处理器610加载并执行预测方法400中的相应步骤；为避免重复，此处不再赘述。
根据本申请的另一方面，本申请还提供了一种编解码系统，包括上文涉及的编码器和解码器。
根据本申请的另一方面，本申请还提供了一种计算机可读存储介质(Memory)，该计算机可读存储介质存储有计算机指令，该计算机指令被计算机设备的处理器读取并执行时，使得计算机设备执行上文涉及的预测方法。
其中，该计算机可读存储介质是解码器或编码器中的记忆设备，用于存放程序和数据。可以理解的是，此处的计算机可读存储介质既可以包括电子设备中的内置存储介质，当然也可以包括电子设备所支持的扩展存储介质。该计算机可读存储介质可以用于提供存储空间，该存储空间可以存储有电子设备的操作系统。此外，在该存储空间中还存放了适于被处理器加载并执行的一条或多条的计算机指令，例如，存放了用于执行上文涉及的预测方法的一条或多条的计算机指令，这些计算机指令可以是一个或多个的计算机程序(包括程序代码)。
根据本申请的另一方面，本申请还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机执行上文涉及的各种可选方式中提供的预测方法。
应当理解，本申请涉及的计算机设备可以是任意一种能够进行数据处理的设备或装置，例如，其包括但不限于：通用计算机、专用计算机、计算机网络、或者其他可编程装置。此外，本申请涉及的计算机指令可以存储在计算机可读存储介质中，也可以在一个计算机可读存储介质和另一个计算机可读存储介质之间进行传输，例如，该计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心传输。
根据本申请的另一方面，本申请还提供了一种码流，该码流可以是利用本申请提供的预测方法对码流进行解码，或利用本申请提供的预测方法生成码流。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元以及流程步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
最后需要说明的是，以上内容，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (20)

  1. 一种基于模板匹配的预测方法，其特征在于，包括：
    基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；
    基于所述第一起始搜索点，利用由所述当前块周围的已重建像素形成的模板，在所述当前块的参考区域内搜索所述当前块的第二矢量；
    基于所述第二矢量，确定所述当前块的预测块。
  2. 根据权利要求1所述的方法，其特征在于，所述第一VP通过第一标识指示，所述第一标识通过码流携带或通过解码端推导确定；或者，所述第一VP为所述VP列表中模板代价最小的VP。
  3. 根据权利要求2所述的方法，其特征在于，所述第一标识通过码流携带时，所述第一标识的码字长度与所述第一VP的模板代价正相关。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，所述确定第一起始搜索点之前，所述方法还包括：
    基于相邻空域VP、时域VP、非相邻空域VP、历史VP、平均VP、零矢量的VP中的至少一项，构建初始列表；
    对所述初始列表中的初始VP执行去重复、矢量调整、模板代价评估、基于模板代价的排序中的至少一项，得到所述VP列表。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点，包括：
    将所述第一矢量所指向的位置，确定为所述第一起始搜索点；或
    对所述第一矢量的至少一个维度进行调整，得到所述第一矢量的调整矢量，以及将所述调整矢量所指向的位置，确定为所述第一起始搜索点。
  6. 根据权利要求5所述的方法，其特征在于，所述对所述第一矢量的至少一个维度进行调整，得到所述第一矢量的第一调整矢量，包括以下至少一项：
    基于所述第一矢量的矢量残差VD，对所述第一矢量的至少一个维度进行调整，得到所述调整矢量；
    基于所述第一矢量确定移动位数，基于所述移动位数对所述第一矢量的至少一个维度进行右移计算，以及基于所述移动位数对右移后的计算结果进行左移计算，得到所述调整矢量；
    将所述第一矢量的至少一个维度，调整为指向距离所述第一矢量所指向的位置最近的整像素位置，得到所述调整矢量；
    基于预设的偏移量，对所述第一矢量的至少一个维度进行偏移，得到所述调整矢量。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述基于所述第一起始搜索点，利用由所述当前块周围的已重建像素形成的模板，在所述当前块的参考区域内搜索所述当前块的第二矢量，包括：
    基于所述第一起始搜索点，搜索至少一个候选搜索点；
    将所述第一矢量调整为指向所述至少一个候选搜索点中模板代价最小的第一最佳搜索点，得到所述第二矢量。
  8. 根据权利要求7所述的方法，其特征在于，所述基于所述第一起始搜索点，搜索至少一个候选搜索点，包括以下至少一项：
    通过遍历搜索候选搜索点的方式，得到所述至少一个候选搜索点；
    基于所述第一起始搜索点和所述当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，直至满足预设条件的情况下，将所述当前迭代过程中得到的模板代价最小的候选搜索点，确定为所述第一最佳搜索点；
    其中，所述预设条件包括以下至少一项：
    所述当前迭代过程中得到的模板代价最小的候选搜索点为所述当前迭代过程使用的起始搜索点；
    已迭代次数大于或等于最大迭代次数；
    已搜索的点大于或等于最大搜索点数。
  9. 根据权利要求8所述的方法，其特征在于，所述基于所述第一起始搜索点和所述当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，包括：
    基于所述第一起始搜索点和所述当前块使用的搜索方法，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点；
    其中，所述搜索方法包括至少一个搜索形状和所述至少一个搜索形状中的每一个搜索形状对应的迭代次数，所述至少一个搜索形状包括所述第一搜索形状。
  10. 根据权利要求8或9所述的方法，其特征在于，所述基于所述第一起始搜索点和所述当前块使用的第一搜索形状，对模板代价最小的候选搜索点进行迭代搜索，得到当前迭代过程中得到的候选搜索点，包括：
    基于所述第一起始搜索点和所述第一搜索形状，对模板代价最小的候选搜索点进行第一迭代过程，得到所述第一迭代过程中得到的候选搜索点，以及基于所述第一迭代过程中得到的候选搜索点中模板代价最小的候选搜索点，对模板代价最小的候选搜索点进行第二迭代过程，得到所述第二迭代过程中得到的候选搜索点；
    在所述第一迭代过程中得到的第一候选搜索点与所述第二迭代过程中得到的第二候选搜索点重叠的情况下，将所述第一候选搜索点的模板代价，确定为所述第二候选搜索点的模板代价；或者
    在所述第一候选搜索点和所述第二候选搜索点重叠、并且所述第二候选搜索点不是所述第二迭代过程使用的起始搜索点的情况下，忽略所述第二候选搜索点，或基于所述第二迭代过程中得到的除所述第二候选搜索点之外的候选搜索点，确定所述第二迭代过程中模板代价最小的候选搜索点。
  11. 根据权利要求7至10中任一项所述的方法，其特征在于，所述至少一个候选搜索点中任意候选搜索点的第一偏移量的绝对值小于或等于第三预设阈值，或所述任意候选搜索点的第二偏移量的绝对值小于或等于第四预设阈值，或所述任意候选搜索点的第一偏移量的绝对值和第二偏移量的绝对值和小于或等于第五预设阈值；
    其中，所述第一偏移量为所述任意候选搜索点在第一维度上相对所述第一起始搜索点的偏移量，所述第二偏移量为所述任意候选搜索点在第二维度上相对所述第一起始搜索点的偏移量。
  12. 根据权利要求7至11中任一项所述的方法，其特征在于，所述得到所述第二矢量之前，所述方法还包括：
    基于所述当前块的多个模板区域，确定所述多个模板区域的代价权重；
    基于所述多个模板区域的代价权重，对所述多个模板区域相对所述至少一个候选搜索点中任意候选搜索点的模板的代价进行加权，得到所述任意候选搜索点的模板代价；
    其中，所述多个模板区域中第一模板区域的代价权重满足以下至少一项：
    所述第一模板区域的代价权重与第一尺寸正相关，所述第一尺寸为所述当前块的与所述第一模板区域相邻的边长的尺寸；
    所述第一模板区域的代价权重与第一距离负相关，所述第一距离为所述第一模板区域与所述当前块之间的距离。
  13. 根据权利要求1至12中任一项所述的方法，其特征在于，所述基于所述第二矢量，确定所述当前块的预测块，包括以下任一项：
    将所述第二矢量指向的第一预测块，确定为所述当前块的预测块；
    对所述第二矢量指向的第一预测块和至少一个矢量指向的预测块进行融合，得到所述当前块的预测块；其中，所述第一预测块的融合权重和所述至少一个矢量中任意矢量指向的预测块的融合权重相等，或所述第一预测块的融合权重和所述任意矢量指向的预测块的融合权重与模板代价负相关；
    其中，所述至少一个矢量包括以下至少一项：
    基于所述第一起始搜索点搜索的除所述第二矢量之外的矢量；
    所述第一VP所包括的除所述第一矢量之外的第三矢量；
    基于所述第三矢量确定第二起始搜索点，以及基于所述第二起始搜索点，利用所述当前块的模板在所述搜索区域内搜索得到的第四矢量；
    基于所述第二起始搜索点搜索的除所述第四矢量之外的矢量；
    基于所述VP列表中除所述第一VP之外的第二VP所包括的第五矢量；
    基于所述第五矢量确定第三起始搜索点，以及基于所述第三起始搜索点，利用所述当前块的模板在所述搜索区域内搜索得到的第六矢量。
  14. 根据权利要求13所述的方法，其特征在于，所述方法还包括以下至少一项：
    基于所述第二起始搜索点，搜索模板代价最小的第二最佳搜索点，基于所述第二最佳搜索点和第一最佳搜索点中模板代价最大的最佳搜索点，搜索模板与第一模板融合后模板代价最小的第三最佳搜索点，以及将所述模板代价最大的最佳搜索点，更新为所述第三最佳搜索点；
    基于所述第二起始搜索点，搜索模板与第一最佳搜索点的模板融合后模板代价最小的候选搜索点，得到所述第二最佳搜索点；
    其中，所述第一模板为所述第一最佳搜索点和所述第二最佳搜索点中模板代价最小的最佳搜索点的模板，所述第一最佳搜索点为基于所述第一起始搜索点搜索的模板代价最小的候选搜索点。
  15. 根据权利要求1至14中任一项所述的方法，其特征在于，所述基于当前块的矢量预测VP列表中的第一矢量预测VP的第一矢量，确定第一起始搜索点，包括：
    在满足以下至少一项的情况下，基于所述第一矢量，确定所述第一起始搜索点：
    第二标识指示利用所述当前块的模板搜索所述当前块的矢量，所述第二标识通过码流携带或通过解码端推导确定；
    通过码流携带的第三标识指示所述当前块所属的当前图像序列、当前图像或当前条带允许利用所述当前块的模板搜索所述当前块的矢量；
    所述当前块的预测模式是帧间预测模式或帧内块复制预测模式；
    当前预测模式是帧间预测模式，且所述当前块的参考帧列表包括前向参考帧和后向参考帧。
  16. 根据权利要求1至14中任一项所述的方法，其特征在于，所述在所述当前块的参考区域内搜索所述当前块的第二矢量，包括：
    使用码流中携带的参数信息，在所述参考区域内搜索所述第二矢量；
    其中，所述参数信息包括以下至少一项：
    迭代搜索使用的最大迭代次数、迭代搜索使用的最大搜索点数、迭代搜索使用的搜索方法、模板代价的计算方式。
  17. 一种基于模板匹配的预测装置，其特征在于，包括：
    第一确定单元，用于基于当前块的矢量预测VP列表中的第一矢量预测VP所包括的第一矢量，确定第一起始搜索点；
    搜索单元，用于基于所述第一起始搜索点，利用由所述当前块周围的已重建像素形成的模板，在所述当前块的参考区域内搜索所述当前块的第二矢量；
    第二确定单元，用于基于所述第二矢量，确定所述当前块的预测块。
  18. 一种电子设备，其特征在于，包括：
    处理器，适于执行计算机程序；
    计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，所述计算机程序被所述处理器执行时，实现权利要求1至16中任一项所述的方法。
  19. 一种计算机可读存储介质，其特征在于，用于存储计算机程序，当所述计算机程序在计算机上运行时，使得计算机执行权利要求1至16中任一项所述的方法。
  20. 一种存储或发送码流的方法，其特征在于，在非暂态计算机可读介质上存储视频码流，所述视频码流基于权利要求1至16中任一项所述的预测方法进行解码，或根据权利要求1至16中任一项所述的预测方法生成。
PCT/CN2025/076891 2024-03-15 2025-02-11 基于模板匹配的预测方法、装置、设备及存储介质 Pending WO2025190011A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020267019607A KR20260113278A (ko) 2024-03-15 2025-02-11 템플릿 매칭 기반 예측 방법 및 장치, 디바이스 그리고 저장 매체

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410306896.XA CN120658883A (zh) 2024-03-15 2024-03-15 基于模板匹配的预测方法、装置、设备及存储介质
CN202410306896.X 2024-03-15

Publications (1)

Publication Number Publication Date
WO2025190011A1 true WO2025190011A1 (zh) 2025-09-18

Family

ID=97003411

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/076891 Pending WO2025190011A1 (zh) 2024-03-15 2025-02-11 基于模板匹配的预测方法、装置、设备及存储介质

Country Status (3)

Country Link
KR (1) KR20260113278A (zh)
CN (1) CN120658883A (zh)
WO (1) WO2025190011A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101170696A (zh) * 2007-11-26 2008-04-30 电子科技大学 一种运动估计方法
CN110545424A (zh) * 2019-08-21 2019-12-06 浙江大华技术股份有限公司 基于mmvd模式的帧间预测方法、视频编码方法及相关装置、设备
US20210006824A1 (en) * 2018-01-08 2021-01-07 Samsung Electronics Co., Ltd. Encoding and decoding method for motion information, and encoding and decoding device for motion information
CN113596475A (zh) * 2021-06-24 2021-11-02 浙江大华技术股份有限公司 图像/视频编码方法、装置、系统及计算机可读存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101170696A (zh) * 2007-11-26 2008-04-30 电子科技大学 一种运动估计方法
US20210006824A1 (en) * 2018-01-08 2021-01-07 Samsung Electronics Co., Ltd. Encoding and decoding method for motion information, and encoding and decoding device for motion information
CN110545424A (zh) * 2019-08-21 2019-12-06 浙江大华技术股份有限公司 基于mmvd模式的帧间预测方法、视频编码方法及相关装置、设备
CN113596475A (zh) * 2021-06-24 2021-11-02 浙江大华技术股份有限公司 图像/视频编码方法、装置、系统及计算机可读存储介质

Also Published As

Publication number Publication date
CN120658883A (zh) 2025-09-16
KR20260113278A (ko) 2026-07-21

Similar Documents

Publication Publication Date Title
CN116320504B (zh) 使用基于历史的运动向量预测进行视频编码的方法和装置
TWI846773B (zh) 用於視訊寫碼之三角運動資訊
CN116847096B (zh) 视频编码方法、计算装置、存储介质和计算机程序产品
TWI843809B (zh) 用於視訊寫碼中具有運動向量差之合併模式之信令傳輸
CN115379237B (zh) 用于视频译码的仿射运动预测
TWI705703B (zh) 於視訊寫碼中針對子區塊推導運動資訊
JP7504925B2 (ja) ビデオコーディングのための組み合わされたインターおよびイントラ予測モード
TWI705694B (zh) 片級內部區塊複製及其他視訊寫碼改善
TW201924345A (zh) 寫碼用於視頻寫碼之仿射預測移動資訊
TW202042552A (zh) 用於照明補償之區塊大小限制
TW201811055A (zh) 使用解耦樹結構之視訊內寫碼
JP2018522466A (ja) ビデオコーディングのための照明補償ステータスを決定するシステムおよび方法
JP2016184936A (ja) 適応型重複ブロック動き補償
US20230017193A1 (en) Video decoding method, video encoding method, electronic device, and storage medium
JP6224851B2 (ja) 低複雑度符号化および背景検出のためのシステムおよび方法
WO2023274360A1 (en) Utilization of recursive prediction unit in video coding
TW201921938A (zh) 具有在用於視訊寫碼之隨機存取組態中之未來參考訊框之可調適圖像群組結構
JP2024513993A (ja) 方法、電子装置、非一時的コンピュータ可読記憶媒体、およびコンピュータプログラム
CN114270860B (zh) 针对仿射模式的自适应运动矢量分辨率
CN112997494B (zh) 用于视频编码的运动矢量存储
JP2016503270A (ja) Ｈｌｓ専用スケーラブル映像コーディングに関する層間基準ピクチャ生成
CN116636218A (zh) 利用多方向帧内预测的视频编解码
WO2023274302A1 (en) Recursive prediction unit in video coding
KR102566569B1 (ko) 인터 예측 방법 및 장치, 비디오 인코더 및 비디오 디코더
WO2020042758A1 (zh) 一种帧间预测的方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25769185

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 1020267019607

Country of ref document: KR

WWP Wipo information: published in national office

Ref document number: 1020267019607

Country of ref document: KR