WO2023123736A1 - 预测方法、装置、设备、系统、及存储介质 - Google Patents

预测方法、装置、设备、系统、及存储介质 Download PDF

Info

Publication number
WO2023123736A1
WO2023123736A1 PCT/CN2022/086453 CN2022086453W WO2023123736A1 WO 2023123736 A1 WO2023123736 A1 WO 2023123736A1 CN 2022086453 W CN2022086453 W CN 2022086453W WO 2023123736 A1 WO2023123736 A1 WO 2023123736A1
Authority
WO
WIPO (PCT)
Prior art keywords
mode
prediction
candidate
template
weight
Prior art date
Application number
PCT/CN2022/086453
Other languages
English (en)
French (fr)
Inventor
王凡
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Priority to CN202280084379.1A priority Critical patent/CN118402239A/zh
Publication of WO2023123736A1 publication Critical patent/WO2023123736A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding

Definitions

  • the present application relates to the technical field of video coding and decoding, and in particular to a prediction method, device, equipment, system, and storage medium.
  • Digital video technology can be incorporated into a variety of video devices, such as digital televisions, smartphones, computers, e-readers, or video players, among others.
  • video devices implement video compression technology to enable more effective transmission or storage of video data.
  • the redundancy in the video can be eliminated or reduced through prediction, and the compression efficiency can be improved.
  • multiple prediction modes are used to predict the current block to obtain the predicted value of the current block.
  • the encoder needs to write the mode information required to predict the current block into the code stream. , which takes up more coding resources and increases the coding cost.
  • Embodiments of the present application provide a prediction method, device, device, system, and storage medium, which determine a weight derivation mode of a current block through K initial prediction modes, thereby reducing coding costs.
  • the present application provides a prediction method applied to a decoder, including:
  • the embodiment of the present application provides a prediction method, including:
  • the present application provides a prediction device, configured to execute the method in the above first aspect or various implementation manners thereof.
  • the prediction device includes a functional unit configured to execute the method in the above first aspect or each implementation manner thereof.
  • the present application provides a prediction device, configured to execute the method in the above second aspect or various implementations thereof.
  • the prediction device includes a functional unit configured to execute the method in the above second aspect or each implementation manner thereof.
  • a video decoder including a processor and a memory.
  • the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory, so as to execute the method in the above first aspect or its various implementations.
  • a sixth aspect provides a video encoder, including a processor and a memory.
  • the memory is used to store a computer program
  • the processor is used to invoke and run the computer program stored in the memory, so as to execute the method in the above second aspect or its various implementations.
  • a video codec system including a video encoder and a video decoder.
  • the video decoder is configured to execute the method in the above first aspect or its various implementations
  • the video encoder is configured to execute the method in the above second aspect or its various implementations.
  • the chip includes: a processor, configured to call and run a computer program from the memory, so that the device installed with the chip executes any one of the above-mentioned first to second aspects or any of the implementations thereof. method.
  • a computer-readable storage medium for storing a computer program, and the computer program causes a computer to execute any one of the above-mentioned first to second aspects or the method in each implementation manner thereof.
  • a computer program product including computer program instructions, the computer program instructions cause a computer to execute any one of the above first to second aspects or the method in each implementation manner.
  • a computer program which, when running on a computer, causes the computer to execute any one of the above-mentioned first to second aspects or the method in each implementation manner thereof.
  • a code stream is provided, and the code stream is generated based on the method in the second aspect above.
  • the weight derivation mode of the current block is determined through K initial prediction modes, thereby enriching the determination method of the weight derivation mode.
  • the encoding end does not need to transmit the index of the weight derivation mode in the code stream, and the decoding end can also determine the weight derivation mode of the current block according to the K initial prediction modes, thereby saving codewords and reducing the encoding cost.
  • FIG. 1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application
  • Fig. 2 is a schematic block diagram of a video encoder involved in an embodiment of the present application
  • Fig. 3 is a schematic block diagram of a video decoder involved in an embodiment of the present application.
  • Fig. 4 is a schematic diagram of weight distribution
  • Fig. 5 is a schematic diagram of weight distribution
  • FIG. 6A is a schematic diagram of inter-frame prediction
  • FIG. 6B is a schematic diagram of weighted inter prediction
  • FIG. 7A is a schematic diagram of intra prediction
  • FIG. 7B is a schematic diagram of intra prediction
  • 8A-8I are schematic diagrams of intra prediction
  • FIG. 9 is a schematic diagram of an intra prediction mode
  • FIG. 10 is a schematic diagram of an intra prediction mode
  • FIG. 11 is a schematic diagram of an intra prediction mode
  • FIG. 12 is a schematic diagram of weighted intra prediction
  • Figure 13 is a schematic diagram of template matching
  • FIG. 14 is a schematic flow chart of a prediction method provided by an embodiment of the present application.
  • Fig. 15 is a schematic diagram when using two prediction modes to predict the current block
  • Fig. 16 is a schematic diagram of template division
  • Fig. 17A is another schematic diagram of template division
  • Fig. 17B is another schematic diagram of template division
  • Fig. 18 is a schematic diagram of template size
  • FIG. 19 is a schematic flow chart of a prediction method provided by an embodiment of the present application.
  • Fig. 20 is a schematic block diagram of a prediction device provided by an embodiment of the present application.
  • Fig. 21 is a schematic block diagram of a prediction device provided by an embodiment of the present application.
  • Fig. 22 is a schematic block diagram of an electronic device provided by an embodiment of the present application.
  • Fig. 23 is a schematic block diagram of a video encoding and decoding system provided by an embodiment of the present application.
  • the application can be applied to the field of image codec, video codec, hardware video codec, dedicated circuit video codec, real-time video codec, etc.
  • the solution of the present application can be combined with audio and video coding standards (audio video coding standard, referred to as AVS), for example, H.264/audio video coding (audio video coding, referred to as AVC) standard, H.265/high efficiency video coding ( High efficiency video coding (HEVC for short) standard and H.266/versatile video coding (VVC for short) standard.
  • the solutions of the present application may operate in conjunction with other proprietary or industry standards, including ITU-TH.261, ISO/IECMPEG-1Visual, ITU-TH.262 or ISO/IECMPEG-2Visual, ITU-TH.263 , ISO/IECMPEG-4Visual, ITU-TH.264 (also known as ISO/IECMPEG-4AVC), including scalable video codec (SVC) and multi-view video codec (MVC) extensions.
  • SVC scalable video codec
  • MVC multi-view video codec
  • FIG. 1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application. It should be noted that FIG. 1 is only an example, and the video codec system in the embodiment of the present application includes but is not limited to what is shown in FIG. 1 .
  • the video codec system 100 includes an encoding device 110 and a decoding device 120 .
  • the encoding device is used to encode (can be understood as compression) the video data to generate a code stream, and transmit the code stream to the decoding device.
  • the decoding device decodes the code stream generated by the encoding device to obtain decoded video data.
  • the encoding device 110 in the embodiment of the present application can be understood as a device having a video encoding function
  • the decoding device 120 can be understood as a device having a video decoding function, that is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120, Examples include smartphones, desktop computers, mobile computing devices, notebook (eg, laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, vehicle-mounted computers, and the like.
  • the encoding device 110 may transmit the encoded video data (such as code stream) to the decoding device 120 via the channel 130 .
  • Channel 130 may include one or more media and/or devices capable of transmitting encoded video data from encoding device 110 to decoding device 120 .
  • channel 130 includes one or more communication media that enable encoding device 110 to transmit encoded video data directly to decoding device 120 in real-time.
  • encoding device 110 may modulate the encoded video data according to a communication standard and transmit the modulated video data to decoding device 120 .
  • the communication medium includes a wireless communication medium, such as a radio frequency spectrum.
  • the communication medium may also include a wired communication medium, such as one or more physical transmission lines.
  • the channel 130 includes a storage medium that can store video data encoded by the encoding device 110 .
  • the storage medium includes a variety of local access data storage media, such as optical discs, DVDs, flash memory, and the like.
  • the decoding device 120 may acquire encoded video data from the storage medium.
  • channel 130 may include a storage server that may store video data encoded by encoding device 110 .
  • the decoding device 120 may download the stored encoded video data from the storage server.
  • the storage server may store the encoded video data and may transmit the encoded video data to the decoding device 120, such as a web server (eg, for a website), a file transfer protocol (FTP) server, and the like.
  • FTP file transfer protocol
  • the encoding device 110 includes a video encoder 112 and an output interface 113 .
  • the output interface 113 may include a modulator/demodulator (modem) and/or a transmitter.
  • the encoding device 110 may include a video source 111 in addition to the video encoder 112 and the input interface 113 .
  • the video source 111 may include at least one of a video capture device (for example, a video camera), a video archive, a video input interface, a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system Used to generate video data.
  • a video capture device for example, a video camera
  • a video archive for example, a video archive
  • a video input interface for example, a video archive
  • video input interface for example, a video input interface
  • computer graphics system used to generate video data.
  • the video encoder 112 encodes the video data from the video source 111 to generate a code stream.
  • Video data may include one or more pictures or a sequence of pictures.
  • the code stream contains the encoding information of an image or image sequence in the form of a bit stream.
  • Encoding information may include encoded image data and associated data.
  • the associated data may include a sequence parameter set (SPS for short), a picture parameter set (PPS for short) and other syntax structures.
  • SPS sequence parameter set
  • PPS picture parameter set
  • An SPS may contain parameters that apply to one or more sequences.
  • a PPS may contain parameters applied to one or more images.
  • the syntax structure refers to a set of zero or more syntax elements arranged in a specified order in the code stream.
  • the video encoder 112 directly transmits encoded video data to the decoding device 120 via the output interface 113 .
  • the encoded video data can also be stored on a storage medium or a storage server for subsequent reading by the decoding device 120 .
  • the decoding device 120 includes an input interface 121 and a video decoder 122 .
  • the decoding device 120 may include a display device 123 in addition to the input interface 121 and the video decoder 122 .
  • the input interface 121 includes a receiver and/or a modem.
  • the input interface 121 can receive encoded video data through the channel 130 .
  • the video decoder 122 is used to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .
  • the display device 123 displays the decoded video data.
  • the display device 123 may be integrated with the decoding device 120 or external to the decoding device 120 .
  • the display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.
  • LCD liquid crystal display
  • plasma display a plasma display
  • OLED organic light emitting diode
  • FIG. 1 is only an example, and the technical solutions of the embodiments of the present application are not limited to FIG. 1 .
  • the technology of the present application may also be applied to one-sided video encoding or one-sided video decoding.
  • Fig. 2 is a schematic block diagram of a video encoder involved in an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression on images, and can also be used to perform lossless compression on images.
  • the lossless compression may be visually lossless compression or mathematically lossless compression.
  • the video encoder 200 can be applied to image data in luminance-chrominance (YCbCr, YUV) format.
  • the YUV ratio can be 4:2:0, 4:2:2 or 4:4:4, Y means brightness (Luma), Cb (U) means blue chroma, Cr (V) means red chroma, U and V are expressed as chroma (Chroma) for describing color and saturation.
  • 4:2:0 means that every 4 pixels have 4 luminance components
  • 2 chroma components YYYYCbCr
  • 4:2:2 means that every 4 pixels have 4 luminance components
  • 4 Chroma component YYYYCbCrCbCr
  • 4:4:4 means full pixel display (YYYYCbCrCbCrCbCrCbCr).
  • the video encoder 200 reads video data, and divides a frame of image into several coding tree units (coding tree units, CTUs) for each frame of image in the video data.
  • CTB can be called “Tree block", “Largest Coding unit” (LCU for short) or “coding tree block” (CTB for short).
  • LCU Large Coding unit
  • CTB coding tree block
  • Each CTU may be associated with a pixel block of equal size within the image. Each pixel may correspond to one luminance (luma) sample and two chrominance (chrominance or chroma) samples.
  • each CTU may be associated with one block of luma samples and two blocks of chroma samples.
  • a CTU size is, for example, 128 ⁇ 128, 64 ⁇ 64, 32 ⁇ 32 and so on.
  • a CTU can be further divided into several coding units (Coding Unit, CU) for coding, and the CU can be a rectangular block or a square block.
  • the CU can be further divided into a prediction unit (PU for short) and a transform unit (TU for short), so that encoding, prediction, and transformation are separated, and processing is more flexible.
  • a CTU is divided into CUs in a quadtree manner, and a CU is divided into TUs and PUs in a quadtree manner.
  • the video encoder and video decoder can support various PU sizes. Assuming that the size of a specific CU is 2N ⁇ 2N, video encoders and video decoders may support 2N ⁇ 2N or N ⁇ N PU sizes for intra prediction, and support 2N ⁇ 2N, 2N ⁇ N, N ⁇ 2N, NxN or similarly sized symmetric PUs for inter prediction. The video encoder and video decoder may also support asymmetric PUs of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prediction.
  • the video encoder 200 may include: a prediction unit 210, a residual unit 220, a transform/quantization unit 230, an inverse transform/quantization unit 240, a reconstruction unit 250, and a loop filter unit 260. Decoded image cache 270 and entropy coding unit 280. It should be noted that the video encoder 200 may include more, less or different functional components.
  • the current block may be called a current coding unit (CU) or a current prediction unit (PU).
  • a predicted block may also be called a predicted image block or an image predicted block, and a reconstructed image block may also be called a reconstructed block or an image reconstructed image block.
  • the prediction unit 210 includes an inter prediction unit 211 and an intra estimation unit 212 . Because there is a strong correlation between adjacent pixels in a video frame, the intra-frame prediction method is used in video coding and decoding technology to eliminate the spatial redundancy between adjacent pixels. Due to the strong similarity between adjacent frames in video, the inter-frame prediction method is used in video coding and decoding technology to eliminate time redundancy between adjacent frames, thereby improving coding efficiency.
  • the inter-frame prediction unit 211 can be used for inter-frame prediction.
  • the inter-frame prediction can include motion estimation (motion estimation) and motion compensation (motion compensation). It can refer to image information of different frames.
  • the inter-frame prediction uses motion information to find reference frames from reference frames. Blocks, generate prediction blocks based on reference blocks to eliminate temporal redundancy; frames used for inter-frame prediction can be P frames and/or B frames, P frames refer to forward prediction frames, and B frames refer to bidirectional prediction frame.
  • Inter-frame prediction uses motion information to find a reference block from a reference frame, and generates a prediction block based on the reference block.
  • the motion information includes the reference frame list where the reference frame is located, the reference frame index, and the motion vector.
  • the motion vector can be an integer pixel or a sub-pixel.
  • the reference frame found according to the motion vector A block of whole pixels or sub-pixels is called a reference block.
  • Some technologies will directly use the reference block as a prediction block, and some technologies will further process the reference block to generate a prediction block. Reprocessing and generating a prediction block based on the reference block can also be understood as taking the reference block as the prediction block and then processing and generating the prediction block of the current block based on the prediction block.
  • the intra-frame estimation unit 212 only refers to the information of the same frame of images to predict the pixel information in the current code image block for eliminating spatial redundancy.
  • a frame used for intra prediction may be an I frame.
  • the intra prediction modes used by HEVC include planar mode (Planar), DC and 33 angle modes, a total of 35 prediction modes.
  • the intra-frame modes used by VVC include Planar, DC and 65 angle modes, with a total of 67 prediction modes.
  • the intra-frame prediction will be more accurate, and it will be more in line with the demand for the development of high-definition and ultra-high-definition digital video.
  • the residual unit 220 may generate a residual block of the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, residual unit 220 may generate a residual block for a CU such that each sample in the residual block has a value equal to the difference between the samples in the pixel blocks of the CU, and the samples in the PUs of the CU. Corresponding samples in the predicted block.
  • Transform/quantization unit 230 may quantize the transform coefficients. Transform/quantization unit 230 may quantize transform coefficients associated with TUs of a CU based on quantization parameter (QP) values associated with the CU. Video encoder 200 may adjust the degree of quantization applied to transform coefficients associated with a CU by adjusting the QP value associated with the CU.
  • QP quantization parameter
  • Inverse transform/quantization unit 240 may apply inverse quantization and inverse transform to the quantized transform coefficients, respectively, to reconstruct a residual block from the quantized transform coefficients.
  • the reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by the prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of the CU in this way, the video encoder 200 can reconstruct the pixel blocks of the CU.
  • the loop filtering unit 260 is used to process the inversely transformed and inversely quantized pixels, compensate for distortion information, and provide better references for subsequent encoded pixels. For example, deblocking filtering operations can be performed to reduce block effect.
  • the loop filtering unit 260 includes a deblocking filtering unit and a sample adaptive compensation/adaptive loop filtering (SAO/ALF) unit, wherein the deblocking filtering unit is used for deblocking, and the SAO/ALF unit Used to remove ringing effects.
  • SAO/ALF sample adaptive compensation/adaptive loop filtering
  • the decoded image buffer 270 may store reconstructed pixel blocks.
  • Inter prediction unit 211 may use reference pictures containing reconstructed pixel blocks to perform inter prediction on PUs of other pictures.
  • intra estimation unit 212 may use the reconstructed pixel blocks in decoded picture cache 270 to perform intra prediction on other PUs in the same picture as the CU.
  • Entropy encoding unit 280 may receive the quantized transform coefficients from transform/quantization unit 230 . Entropy encoding unit 280 may perform one or more entropy encoding operations on the quantized transform coefficients to generate entropy encoded data.
  • Fig. 3 is a schematic block diagram of a video decoder involved in an embodiment of the present application.
  • the video decoder 300 includes: an entropy decoding unit 310 , a prediction unit 320 , an inverse quantization/transformation unit 330 , a reconstruction unit 340 , a loop filter unit 350 and a decoded image buffer 360 . It should be noted that the video decoder 300 may include more, less or different functional components.
  • the video decoder 300 can receive code streams.
  • the entropy decoding unit 310 may parse the codestream to extract syntax elements from the codestream. As part of parsing the codestream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the codestream.
  • the prediction unit 320 , the inverse quantization/transformation unit 330 , the reconstruction unit 340 and the loop filter unit 350 can decode video data according to the syntax elements extracted from the code stream, that is, generate decoded video data.
  • the prediction unit 320 includes an inter prediction unit 321 and an intra estimation unit 322 .
  • Intra estimation unit 322 may perform intra prediction to generate a predictive block for a PU. Intra-estimation unit 322 may use an intra-prediction mode to generate a predictive block for a PU based on pixel blocks of spatially neighboring PUs. Intra estimation unit 322 may also determine the intra prediction mode for the PU from one or more syntax elements parsed from the codestream.
  • the inter prediction unit 321 can construct the first reference picture list (list 0) and the second reference picture list (list 1) according to the syntax elements parsed from the codestream. Furthermore, if the PU is encoded using inter prediction, entropy decoding unit 310 may parse the motion information for the PU. Inter prediction unit 321 may determine one or more reference blocks for the PU according to the motion information of the PU. Inter prediction unit 321 may generate a prediction block for a PU based on one or more reference blocks of the PU.
  • Inverse quantization/transform unit 330 may inverse quantize (ie, dequantize) transform coefficients associated with a TU. Inverse quantization/transform unit 330 may use QP values associated with CUs of the TU to determine the degree of quantization.
  • inverse quantization/transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.
  • Reconstruction unit 340 uses the residual blocks associated with the TUs of the CU and the prediction blocks of the PUs of the CU to reconstruct the pixel blocks of the CU. For example, the reconstruction unit 340 may add the samples of the residual block to the corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain the reconstructed image block.
  • Loop filtering unit 350 may perform deblocking filtering operations to reduce blocking artifacts of pixel blocks associated with a CU.
  • Video decoder 300 may store the reconstructed picture of the CU in decoded picture cache 360 .
  • the video decoder 300 may use the reconstructed picture in the decoded picture buffer 360 as a reference picture for subsequent prediction, or transmit the reconstructed picture to a display device for presentation.
  • the basic process of video encoding and decoding is as follows: at the encoding end, a frame of image is divided into blocks, and for the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block of the current block.
  • the residual unit 220 may calculate a residual block based on the predicted block and the original block of the current block, that is, a difference between the predicted block and the original block of the current block, and the residual block may also be referred to as residual information.
  • the residual block can be transformed and quantized by the transformation/quantization unit 230 to remove information that is not sensitive to human eyes, so as to eliminate visual redundancy.
  • the residual block before being transformed and quantized by the transform/quantization unit 230 may be called a time domain residual block, and the time domain residual block after being transformed and quantized by the transform/quantization unit 230 may be called a frequency residual block or a frequency-domain residual block.
  • the entropy coding unit 280 receives the quantized variation coefficients output by the variation quantization unit 230 , and may perform entropy coding on the quantized variation coefficients to output a code stream.
  • the entropy coding unit 280 can eliminate character redundancy according to the target context model and the probability information of the binary code stream.
  • the entropy decoding unit 310 can analyze the code stream to obtain the prediction information of the current block, the quantization coefficient matrix, etc., and the prediction unit 320 uses intra prediction or inter prediction for the current block based on the prediction information to generate a prediction block of the current block.
  • the inverse quantization/transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to perform inverse quantization and inverse transformation on the quantization coefficient matrix to obtain a residual block.
  • the reconstruction unit 340 adds the predicted block and the residual block to obtain a reconstructed block.
  • the reconstructed blocks form a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or based on the block to obtain a decoded image.
  • the encoding end also needs similar operations to the decoding end to obtain the decoded image.
  • the decoded image may also be referred to as a reconstructed image, and the reconstructed image may be a subsequent frame as a reference frame for inter-frame prediction.
  • the block division information determined by the encoder as well as mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, etc., are carried in the code stream when necessary.
  • the decoding end analyzes the code stream and analyzes the existing information to determine the same block division information as the encoding end, prediction, transformation, quantization, entropy coding, loop filtering and other mode information or parameter information, so as to ensure the decoding image obtained by the encoding end It is the same as the decoded image obtained by the decoder.
  • the above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. This application is applicable to the block-based hybrid coding framework.
  • the basic process of the video codec but not limited to the framework and process.
  • the current block may be a current coding unit (CU), a current prediction unit (PU), or the like. Due to the need for parallel processing, images can be divided into slices, etc., and slices in the same image can be processed in parallel, that is to say, there is no data dependence between them. And "frame” is a commonly used term, which can generally be understood as a frame is an image. The frames mentioned in the application can also be replaced by images or slices, etc.
  • VVC Versatile Video Coding
  • ADP Angular Weighted Prediction
  • the traditional unidirectional prediction only finds a reference block with the same size as the current block
  • the traditional bidirectional prediction uses two reference blocks with the same size as the current block
  • the pixel value of each point in the predicted block is The average value of the corresponding positions of the two reference blocks, that is, all points of each reference block account for 50% of the proportion.
  • Bidirectional weighted prediction makes the proportions of two reference blocks different, for example, all points in the first reference block account for 75% of the proportion, and all points in the second reference block account for 25% of the proportion. But all points in the same reference block have the same scale.
  • DMVR Decoder sideMotion Vector Refinement
  • BIO bi-directional optical flow
  • GPM or AWP will also Use two reference blocks with the same size as the current block, but some pixel positions use 100% the pixel values corresponding to the first reference block, some pixel positions 100% use the pixel values corresponding to the second reference block, and In the boundary area, the pixel values of the corresponding positions of the two reference blocks are used according to a certain ratio.
  • GPM or AWP uses two reference blocks that are different in size from the current block, that is, each takes a required part as a reference block. That is, the part whose weight is not 0 is used as a reference block, and the part whose weight is 0 is eliminated.
  • FIG. 4 is a schematic diagram of weight distribution, as shown in FIG. 4 , which shows a schematic diagram of weight distribution of multiple division modes of a GPM on a 64 ⁇ 64 current block provided by an embodiment of the present application, wherein, There are 64 division modes in GPM.
  • Fig. 5 is a schematic diagram of weight distribution. As shown in Fig. 5, it shows a schematic diagram of weight distribution of various division modes of an AWP on a 64 ⁇ 64 current block provided by an embodiment of the present application, wherein there are 56 a division mode.
  • the black area indicates that the weight value of the corresponding position of the first reference block is 0%
  • the white area indicates that the weight value of the corresponding position of the first reference block is 100%
  • the gray area indicates that the weight value of the corresponding position of the first reference block is 100%.
  • the area indicates a weight value corresponding to the first reference block with a weight value greater than 0% and less than 100% according to the color depth, and the weight value corresponding to the second reference block is 100% minus the first The weight value of the corresponding position of a reference block.
  • GPM determines the angle and offset according to each mode, and then calculates the weight matrix of each mode.
  • AWP first makes a one-dimensional weight line, and then uses a method similar to intra-frame angle prediction to spread the one-dimensional weight line across the entire matrix.
  • GPM and AWP achieve the predicted non-rectangular division effect without division.
  • GPM and AWP use a mask of the weights of two reference blocks, ie the weight map mentioned above. This mask determines the weight of the two reference blocks when generating the prediction block, or it can be simply understood as part of the position of the prediction block comes from the first reference block and part of the position comes from the second reference block, and the transition area (blending area) weighted by the corresponding positions of the two reference blocks, so as to make the transition smoother.
  • GPM and AWP do not divide the current block into two CUs or PUs according to the dividing line, so the transformation, quantization, inverse transformation, and inverse quantization of the residual after prediction are also processed by taking the current block as a whole.
  • GPM simulates the division of geometry, more precisely the division of predictions, using weight matrices.
  • two predictors are required, and each predictor is determined by one unidirectional motion information. These two pieces of unidirectional motion information come from a motion information candidate list, for example, from a merge motion information candidate list (mergeCandList).
  • GPM uses two indexes in the code stream to determine 2 unidirectional motion information from mergeCandList.
  • Inter prediction uses motion information to represent "motion".
  • Basic motion information includes reference frame (reference frame) (or reference image (reference picture)) information and motion vector (MV, motion vector) information.
  • the commonly used bidirectional prediction uses two reference blocks to predict the current block. 2 reference blocks can use a forward reference block and a backward reference block. Optionally, both are forward or both are backward are allowed.
  • the so-called forward means that the time corresponding to the reference frame is before the current frame
  • the backward means that the time corresponding to the reference frame is after the current frame.
  • the forward direction refers to the position of the reference frame in the video before the current frame
  • the backward direction refers to the position of the reference frame in the video after the current frame.
  • the POC (picture order count) of the forward reference frame is smaller than the POC of the current frame
  • the POC of the backward reference frame is greater than the POC of the current frame.
  • two sets of reference frame information and motion vector information are required. Each of them can be understood as a one-way motion information, and the combination of these two groups forms a two-way motion information.
  • the unidirectional motion information and the bidirectional motion information can use the same data structure, but the two sets of reference frame information and the motion vector information of the bidirectional motion information are valid, and one of the reference frames of the unidirectional motion information information and motion vector information is invalid.
  • two reference frame lists are supported, denoted as RPL0 and RPL1, where RPL is an abbreviation for Reference Picture List.
  • RPL is an abbreviation for Reference Picture List.
  • the P slice can only use RPL0, and the B slice can use RPL0 and RPL1.
  • the codec finds a certain reference frame through the reference frame index.
  • motion information is represented by a reference frame index and a motion vector.
  • the reference frame index refIdxL0 corresponding to reference frame list 0 the motion vector mvL0 corresponding to reference frame list 0, the reference frame index refIdxL1 corresponding to reference frame list 1, and the motion corresponding to reference frame list 1 Vector mvL0.
  • the reference frame index corresponding to the reference frame list 0 and the reference frame index corresponding to the reference frame list 1 can be understood as the above-mentioned reference frame information.
  • two flag bits are used to respectively indicate whether to use the motion information corresponding to the reference frame list 0 and whether to use the motion information corresponding to the reference frame list 0, respectively marked as predFlagL0 and predFlagL1.
  • predFlagL0 and predFlagL1 indicate whether the above-mentioned one-way motion information is valid or not.
  • the data structure of motion information is not explicitly mentioned, it uses the reference frame index corresponding to each reference frame list, the motion vector and the "valid or not" flag to represent the motion information. In some standard texts, the motion information does not appear, but the motion vector is used. It can also be considered that the reference frame index and the flag of whether to use the corresponding motion information are attached to the motion vector. In this application, "motion information” is still used for convenience of description, but it should be understood that "motion vector” may also be used for description.
  • the motion information used by the current block can be saved.
  • Subsequent encoded and decoded blocks of the current frame can use motion information of previously encoded and decoded blocks, such as adjacent blocks, according to the adjacent positional relationship. This utilizes the correlation in the spatial domain, so this encoded and decoded motion information is called motion information in the spatial domain.
  • the motion information used by each block of the current frame can be preserved.
  • Subsequent codec frames can use the motion information of previous codec frames according to the reference relationship. This utilizes the correlation in the time domain, so the motion information of the encoded and decoded frames is called the motion information in the time domain.
  • the storage method of the motion information used by each block of the current frame usually uses a fixed-size matrix, such as a 4x4 matrix, as a minimum unit, and each minimum unit independently stores a set of motion information.
  • a fixed-size matrix such as a 4x4 matrix
  • each minimum unit independently stores a set of motion information.
  • every time a block is encoded and decoded the smallest units corresponding to its position can store the motion information of this block.
  • the motion information corresponding to the position can be found directly according to the position. If a 16x16 block uses traditional unidirectional prediction, then all 4x4 minimum units corresponding to this block store the motion information of this unidirectional prediction.
  • a block uses GPM or AWP, then all the smallest units corresponding to this block will determine each smallest unit according to the GPM or AWP mode, the first motion information, the second motion information, and the position of each smallest unit Stored exercise information.
  • One method is that if all the 4x4 pixels corresponding to a minimum unit come from the first motion information, then this minimum unit stores the first motion information, and if all the 4x4 pixels corresponding to a minimum unit come from the second motion information , then this smallest unit stores the second motion information.
  • AWP will select one of the motion information for storage; the GPM approach is if the two motion information points to different references frame list, then combine them into two-way motion information storage, otherwise just store the second motion information.
  • the above mergeCandList is constructed according to spatial motion information, time domain motion information, history-based motion information, and some other motion information.
  • mergeCandList uses positions 1 to 5 in FIG. 6A to derive spatial motion information, and uses positions 6 or 7 in FIG. 6A to derive time domain motion information.
  • History-based motion information is to add the motion information of this block to a first-in-first-out list every time a block is encoded and decoded. The adding process may require some checks, such as whether it is duplicated with the existing motion information in the list. In this way, the motion information in this history-based list can be referred to when encoding and decoding the current block.
  • the syntax description about GPM is as shown in Table 1:
  • the current block may use CIIP or GPM. If the current block does not use CIIP, then it uses GPM, which is the content shown in the syntax "if(!ciip_flag[x0][y0])" in Table 1.
  • GPM needs to transmit three pieces of information in the code stream, namely merge_gpm_partition_idx, merge_gpm_idx0, and merge_gpm_idx1.
  • x0, y0 are used to determine the coordinates (x0, y0) of the luminance pixel in the upper left corner of the current block relative to the luminance pixel in the upper left corner of the image.
  • merge_gpm_partition_idx determines the partition shape of the GPM, as shown above, it is "analog partition", and merge_gpm_partition_idx is the weight derivation mode or the index of the weight derivation mode mentioned in the embodiment of this application.
  • merge_gpm_idx0 is the index value of the first motion information in the candidate list
  • merge_gpm_idx1 is the index value of the second motion information in the candidate list. If the candidate list length (MaxNumGpmMergeCand)>2, you need to pass merge_gpm_idx1, otherwise you can directly determine.
  • the decoding process of GPM includes the following steps:
  • the information input in the decoding process includes: the coordinates (xCb, yCb) of the brightness position of the upper left corner of the current block relative to the upper left corner of the image, the width cbWidth of the brightness component of the current block, the height cbHeight of the brightness component of the current block, and the brightness of 1/16 pixel accuracy
  • motion information may be represented by a combination of a motion vector, a reference frame index and a prediction list flag.
  • 2 reference frame lists are supported, each reference frame list may have multiple reference frames.
  • the unidirectional prediction uses only one reference block of one reference frame in one of the reference frame lists as a reference, and the bidirectional prediction uses one reference block of each reference frame in each of the two reference frame lists as a reference.
  • GPM uses 2 unidirectional forecasts.
  • a in the above mvA and mvB, mvCA and mvCB, refIdxA and refIdxB, predListFlagA and predListFlagB can be understood as the first prediction mode, and B can be understood as the second prediction mode.
  • predListFlagX indicates whether X uses the first reference frame list or the second reference frame list
  • refIdxX indicates the reference frame index in the reference frame list used by X
  • mvX indicates the brightness motion used by X Vector
  • mvCX represents the chroma motion vector used by X.
  • the information output by the decoding process includes: (cbWidth)X(cbHeight) luminance prediction sample matrix predSamplesL; (cbWidth/SubWidthC)X(cbHeight/SubHeightC) Cb chrominance component prediction sample matrix, if necessary; (cbWidth/SubWidthC) X(cbHeight/SubHeightC) predictor sample matrix for the Cr chroma component, if required.
  • the luma component is used as an example below, and the processing of the chrominance component is similar to that of the luma component.
  • predSamplesLAL and predSamplesLBL are (cbWidth)X(cbHeight), which are prediction sample matrices made according to two prediction modes.
  • predSamplesL is derived as follows: predSamplesLAL and predSamplesLBL are determined according to luma motion vectors mvA and mvB, chrominance motion vectors mvCA and mvCB, reference frame indices refIdxA and refIdxB, and prediction list flags predListFlagA and predListFlagB, respectively. That is, the prediction is performed according to the motion information of the two prediction modes respectively, and the detailed process will not be repeated here.
  • GPM is a merge mode, and it can be considered that the two prediction modes of GPM are both merge modes.
  • the process of generating the predicted sample matrix of GPM by one component is divided into a sub-process, that is, GPM
  • the Weighted sample prediction process for geometric partitioning mode the three components will call this process, but the parameters of the call are different, here only the brightness component is used as an example.
  • nCbW is set to cbWidth
  • nCbH is set to cbHeight
  • the prediction sample matrices predSamplesLAL and predSamplesLBL made by the two prediction modes, and angleIdx and distanceIdx are used as input.
  • the weighted forecast derivation process of GPM includes the following steps:
  • the input of this process is: the width nCbW of the current block, the height nCbH of the current block; two (nCbW)X(nCbH) prediction sample matrices predSamplesLA and predSamplesLB; the "division" angle index variable angleIdx of GPM; the distance index variable of GPM distanceIdx; component index variable cIdx.
  • This example uses luminance as an example, so the cIdx above is 0, indicating a luminance component.
  • the output of this process is: (nCbW)X(nCbH) GPM prediction sample matrix pbSamples.
  • nW, nH, shift1, offset1, displacementX, displacementY, partFlip and shiftHor are derived as follows:
  • offsetY ((-nH)>>1)+(angleIdx ⁇ 16? (distanceIdx*nH)>>3:-((distanceIdx*nH)>>3)).
  • offsetX ((-nW)>>1)+(angleIdx ⁇ 16? (distanceIdx*nW)>>3:-((distanceIdx*nW)>>3),
  • variable wValue representing the weight of the prediction sample at the current position is derived as follows, wValue is the weight of the prediction value predSamplesLA[x][y] of the prediction matrix of the first prediction mode at point (x, y), and (8-wValue) That is, the weight of the predicted value predSamplesLB[x][y] of the prediction matrix of the first prediction mode at point (x, y).
  • the distance matrix disLut is determined according to Table 3:
  • weightIdx (((xL+offsetX) ⁇ 1)+1)*disLut[displacementX]+(((yL+offsetY) ⁇ 1)+1)*disLut[displacementY],
  • weightIdxL partFlip? 32+weightIdx:32–weightIdx,
  • pbSamples[x][y] Clip3(0,(1 ⁇ BitDepth)-1,(predSamplesLA[x][y]*wValue+predSamplesLB[x][y]*(8-wValue)+offset1)>> shift1).
  • a weight value is derived for each position of the current block, and then a predicted value pbSamples[x][y] of a GPM is calculated. Because this way the weight wValue does not have to be written in the form of a matrix, but it can be understood that if the wValue of each position is saved in a matrix, then it is a weight matrix. Calculate the weight of each point separately and weight it to get the predicted value of GPM, or calculate all the weights and then weight them uniformly to get the predicted sample matrix of GPM. The principle is the same.
  • the use of the weight matrix in many descriptions in this application is to make the expression easier to understand, and it is more intuitive to use the weight matrix to draw pictures. In fact, it can also be described according to the weight of each position.
  • the weight matrix export mode can also be called the weight export mode.
  • the decoding process of GPM can be expressed as: analyze the code stream, determine whether the current block uses GPM technology; if the current block uses GPM technology, determine the weight derivation mode (or "division" mode or weight matrix derivation mode), and the first motion information and the second motion information. Determine the first prediction block according to the first motion information, determine the second prediction block according to the second motion information, determine the weight matrix according to the weight matrix derivation mode, and determine the prediction of the current block according to the first prediction block and the second prediction block and the weight matrix piece.
  • GPM or AWP belongs to a prediction technology, and GPM or AWP needs to transmit a flag (flag) of whether GPM or AWP is used in the code stream, and the flag can indicate whether the current block is Use GPM or AWP.
  • the encoder needs to transmit the specific mode used in the code stream, that is, one of the 64 division modes of GPM, or one of the 56 division modes of AWP; and the index value of two unidirectional motion information. That is to say, for the current block, the decoder can obtain information about whether GPM or AWP is used by parsing the code stream.
  • the decoder can parse out the prediction mode parameters of GPM or AWP and two motion information Index value, for example, the current block can be divided into two partitions, then the first index value corresponding to the first partition and the second index value corresponding to the second partition can be parsed out.
  • the prediction mode parameters under GPM will be transmitted in the code stream, such as the specific division mode of GPM; usually, GPM includes 64 division modes.
  • GPM the specific division mode of GPM
  • AWP the prediction mode parameters under AWP will be transmitted in the code stream, such as the specific division mode of AWP; usually, AWP includes 56 division modes.
  • the current implementation method is to construct a unidirectional motion information candidate list on the encoder side by using the relevant information of the coded/decoded part before the current block, select unidirectional motion information from the unidirectional motion information candidate list, and combine the two unidirectional motion information Write the code stream to the index value (index) of the motion information in the unidirectional motion information candidate list.
  • the same method is adopted on the decoder side, that is, a unidirectional motion information candidate list is constructed using the relevant information of the decoded part before the current block, and the unidirectional motion information candidate list must be the same as the candidate list constructed on the encoder side. In this way, the index values of the two unidirectional motion information are parsed from the code stream, and then the two unidirectional motion information are found out from the unidirectional motion information candidate list, which is the two unidirectional motion information to be used by the current block.
  • the unidirectional motion information described in this application may include: motion vector information, that is, the value of (x, y), and corresponding reference frame information, that is, the reference frame list and the reference frame index in the reference frame list value.
  • motion vector information that is, the value of (x, y)
  • corresponding reference frame information that is, the reference frame list and the reference frame index in the reference frame list value.
  • One representation is to record the reference frame index values of two reference frame lists, where the reference frame index value corresponding to one reference frame list is valid, such as 0, 1, 2, etc.; the reference frame index value corresponding to the other reference frame list is Invalid, i.e. -1.
  • the reference frame list with valid reference frame index value is the reference frame list used by the motion information of the current block, and the corresponding reference frame can be found from the reference frame list according to the reference frame index value.
  • Each reference frame list has a corresponding motion vector, the motion vector corresponding to the valid reference frame list is valid, and the motion vector corresponding to the invalid reference frame list is invalid.
  • the decoder can find the required reference frame through the reference frame information in the unidirectional motion information, and can find the reference block in the reference frame according to the position of the current block and the value of the motion vector (x, y), and then determine the current block The inter-frame prediction value of .
  • the intra-frame prediction method uses coded and decoded reconstructed pixels surrounding the current block as reference pixels to predict the current block.
  • Figure 7A is a schematic diagram of intra-frame prediction. As shown in Figure 7A, the size of the current block is 4x4, and the pixels on the left row and upper column of the current block are reference pixels of the current block, and intra-frame prediction uses these reference pixels to predict the current block .
  • These reference pixels may all be available, that is, all have been encoded and decoded. Some parts may also be unavailable, for example, the current block is the leftmost of the whole frame, then the reference pixel on the left of the current block is unavailable.
  • the lower left part of the current block has not been encoded and decoded, so the reference pixel at the lower left is also unavailable.
  • the available reference pixel or some value or some method can be used for filling, or no filling is performed.
  • FIG. 7B is a schematic diagram of intra prediction.
  • the multiple reference line intra prediction method (Multiple reference line, MRL) can use more reference pixels to improve the encoding and decoding efficiency, for example, using 4 reference lines/ is listed as the reference pixel of the current block.
  • FIG. 8A-5I are schematic diagrams of intra-frame prediction.
  • intra-frame prediction for 4x4 blocks in H.264 mainly includes 9 modes. Among them, mode 0 as shown in FIG. 8A copies the pixels above the current block to the current block in the vertical direction as the prediction value, and mode 1 as shown in FIG. 8B copies the reference pixel on the left to the current block in the horizontal direction as the prediction value.
  • the mode 2 direct current DC shown in Figure 8C uses the average value of the 8 points A ⁇ D and I ⁇ L as the predicted value of all points, and the modes 3 ⁇ 8 shown in Figure 8D-5I respectively press a certain angle Copy the reference pixel to the corresponding position of the current block, because some positions of the current block cannot exactly correspond to the reference pixel, it may be necessary to use the weighted average of the reference pixel, or the sub-pixel of the interpolated reference pixel.
  • FIG. 9 is a schematic diagram of intra-frame prediction modes.
  • the intra-frame prediction modes used by HEVC include Planar, DC and 33 angle modes, a total of 35 prediction modes.
  • FIG. 10 is a schematic diagram of an intra-frame prediction mode.
  • the intra-frame modes used by VVC include Planar, DC and 65 angle modes, a total of 67 prediction modes.
  • Fig. 11 is a schematic diagram of intra prediction modes. As shown in Fig. 11, AVS3 uses DC, Planar, Bilinear and 63 angle modes, a total of 66 prediction modes.
  • the multiple intraprediction filter (MIPF) in AVS3 uses different filters to generate prediction values for different block sizes. For pixels at different positions in the same block, a filter is used to generate a prediction value for pixels that are closer to the reference pixel, and another filter is used to generate a prediction value for pixels far from the reference pixel.
  • MIPF multiple intraprediction filter
  • a technique for filtering predicted pixels, such as intraprediction filter (IPF) in AVS3, can use reference pixels to filter predicted values.
  • the most probable mode list (MostprobableModes List, MPM) intra-mode encoding technology can be used to improve the encoding and decoding efficiency.
  • MPM most probable mode list
  • intra-frame prediction mode of the surrounding encoded and decoded blocks and the intra-frame prediction mode derived from the intra-frame prediction mode of the surrounding encoded and decoded blocks, such as adjacent modes, and some intra-frame predictions that are commonly used or have a relatively high probability of use Modes, such as DC, Planar, Bilinear mode, etc., constitute a mode list.
  • Intra prediction modes that refer to surrounding coded blocks take advantage of spatial correlation. Because the texture will have a certain continuity in space. MPM can be used as the prediction of the intra prediction mode. That is to say, the probability that the current block uses MPM is higher than that of not using MPM. Therefore, during binarization, fewer codewords will be used for MPM, thereby saving overhead and improving encoding and decoding efficiency.
  • GPM combines two inter prediction blocks with a weight matrix. In fact it can be extended to combine two arbitrary prediction blocks. Such as two inter prediction blocks, two intra prediction blocks, one inter prediction block and one intra prediction block. Even in screen content coding, IBC (intra block copy) or palette prediction blocks can be used as one or two prediction blocks.
  • IBC intra block copy
  • palette prediction blocks can be used as one or two prediction blocks.
  • the prediction mode can be understood as information according to which the codec can generate a prediction block of the current block.
  • the prediction mode may be a certain intra-frame prediction mode, such as DC, Planar, various intra-frame angle prediction modes, and the like.
  • some or some auxiliary information can also be superimposed, such as the optimization method of the reference pixel in the frame, the optimization method (such as filtering) after the preliminary prediction block is generated, and the like.
  • the prediction mode can be skip (skip) mode, merge (merge) mode or MMVD (merge with motion vector difference, merge with motion vector difference) mode, or AMVP (advanced motion vector predition, Advanced motion vector pre-prediction), which can be unidirectional prediction or bidirectional prediction or multi-hypothesis prediction.
  • the inter-frame prediction mode uses unidirectional prediction, it must be able to determine a motion information, which is a unidirectional motion information, and the prediction block can be determined according to the motion information.
  • the inter-frame prediction mode uses bidirectional prediction, it must be able to determine one bidirectional motion information or two unidirectional motion information, and the prediction block can be determined according to the motion information.
  • the inter-frame prediction mode uses multi-hypothesis prediction, it must be able to determine multiple unidirectional motion information, and the prediction block can be determined according to the motion information.
  • skip, merge, and common inter mode can all support unidirectional prediction, bidirectional prediction or multi-hypothesis prediction.
  • a prediction mode is an inter-frame prediction mode, it can determine motion information, and a prediction block can be determined according to the motion information.
  • the template matching method can be used on the basis of skip mode and merge mode, MMVD mode, and ordinary inter mode. Such a prediction mode can still be called skip mode and merge mode, MMVD mode, ordinary inter mode or skip using template matching. mode, the merge mode using template matching, the MMVD mode using template matching, and the normal inter mode using template matching.
  • MMVD can be considered as a special merge mode, which indicates some specific MVDs through some flag bits, and these specific MVDs have only several possible preset values.
  • An example is the MMVD mode in VVC, which uses mmvd_direction_idx to indicate the direction of the MVD, and the possible values of mmvd_direction_idx are 0, 1, 2, 3.
  • mmvd_distance_idx 0 indicates that the horizontal component of MMVD is positive, and the vertical direction is 0; 1 indicates that of MMVD The horizontal component is negative, and the vertical direction is 0; 2 means that the horizontal component of MMVD is 0, and the vertical direction is positive; 3 means that the horizontal component of MMVD is 0, and the vertical direction is negative.
  • Use mmvd_distance_idx to represent the absolute value of the positive or negative value above.
  • the possible values of mmvd_distance_idx are 0 to 7.
  • the MVD of the ordinary inter mode can theoretically represent any possible MVD within a valid range.
  • the information that GPM needs to determine can be expressed as one weight derivation mode and two prediction modes.
  • the weight derivation mode is used to determine the weight matrix or weight, and the two prediction modes respectively determine a prediction block or a prediction value.
  • Weight export mode is also called partition mode in some places. But because it is an analog division, this application is more commonly called the weight derivation mode.
  • the two prediction modes may come from the same or different prediction methods, where the prediction methods include but not limited to intra prediction, inter prediction, IBC, and palette.
  • a specific concrete example is as follows: If the current block uses GPM. This example is used in inter-coded blocks, allowing the use of merge mode in intra prediction and inter prediction. As shown in Table 4, add a syntax element intra_mode_idx to indicate which prediction mode is an intra prediction mode.
  • intra_mode_idx 0
  • intra_mode_idx 1
  • intra_mode_idx 1
  • intra_mode_idx 2
  • intra_mode_idx 2
  • intra_mode_idx 3
  • both prediction modes are intra prediction modes, that is, mode0IsInter is 0, and mode0IsInter is 0.
  • the decoding process of GPM can be expressed as: analyze the code stream, determine whether the current block uses GPM technology; if the current block uses GPM technology, determine the weight derivation mode (or "division" mode or weight matrix derivation mode), and the first intra prediction mode and the second intra prediction mode.
  • the first prediction block is determined according to the first intra prediction mode
  • the second prediction block is determined according to the second intra prediction mode
  • the weight matrix is determined according to the weight matrix derivation mode
  • the weight matrix is determined according to the first prediction block and the second prediction block and the weight matrix The predicted block for the current block.
  • template matching was first used in inter-frame prediction, which uses the correlation between adjacent pixels to use some areas around the current block as templates.
  • inter-frame prediction uses the correlation between adjacent pixels to use some areas around the current block as templates.
  • the left and upper sides of the current block have been encoded and decoded according to the encoding order.
  • the existing hardware decoder is implemented, it may not be guaranteed that when the current block starts decoding, its left and upper sides have been decoded.
  • the inter-frame block such as the inter-frame coded block in HEVC.
  • the surrounding reconstructed pixels are not needed when predicting a block, so the prediction process of an inter block can be performed in parallel.
  • the intra-coded block must require the reconstructed pixels on the left side and the upper side as reference pixels.
  • the left side and the upper side are available, that is to say, it is achievable to adjust the hardware design accordingly.
  • the right side and the bottom side are not available under the encoding order of current standards such as VVC.
  • the left and upper rectangular areas of the current block are set as templates.
  • the height of the left template part is generally the same as the height of the current block, and the width of the upper template part is generally the same as the width of the current block. The same, of course, can also be different.
  • the so-called matching degree can be measured by some distortion costs, such as SAD (sum of absolute difference), SATD (sum of absolute transformed difference ), the transformation used by SATD is Hadamard transformation, MSE (mean-square error), etc.
  • SAD sum of absolute difference
  • SATD sum of absolute transformed difference
  • MSE mean-square error
  • the method of template matching may not be applicable to all blocks, so some methods can be used to determine whether the current block uses the above method of template matching, such as using a control switch in the current block to indicate whether to use template matching.
  • a name for this template matching method is DMVD (decoder side motion vector derivation).
  • Both the encoder and the decoder can use templates to search to derive motion information or find better motion information based on the original motion information. And it does not need to transmit specific motion vectors or motion vector differences, but both the encoder and the decoder perform the same rule search to ensure the consistency of encoding and decoding.
  • the method of template matching can improve the compression performance, but it also needs to "search" in the decoder, which brings a certain complexity of the decoder.
  • the method of template matching can also be used in intra frames, for example, using templates to determine the intra prediction mode.
  • the current block you can also use the area within a certain range on the top and left of the current block as a template, for example, the left rectangular area and the upper rectangular area as shown in the figure above.
  • the reconstructed pixels in the template are available when encoding and decoding the current block.
  • This process can roughly be described as determining a set of candidate intra-frame prediction modes for the current block, and the candidate intra-frame prediction modes constitute a subset of all available intra-frame prediction modes.
  • the candidate intra-frame prediction mode may be a complete set of all available intra-frame prediction modes.
  • the set of candidate intra prediction modes can be determined according to the MPM or some rules, such as equidistant screening. Calculate the cost of each candidate intra prediction mode on the template, such as SAD, SATD, MSE, etc. Use this mode to predict on the template to make a prediction block, and use the prediction block and the reconstruction block of the template to calculate the cost. A mode with a low cost may better match the template. Using the similarity between adjacent pixels, the intra prediction mode that performs well on the template may also be the intra prediction mode that performs well on the current block. Select one or several low-cost models. Of course, the above two steps can be repeated.
  • the set of candidate intra-frame prediction modes After selecting one or several low-cost modes, determine the set of candidate intra-frame prediction modes again, and then calculate the cost for the newly determined set of candidate intra-frame prediction modes. , select one or several low-cost models. This can also be understood as rough selection and fine selection.
  • the finally selected intra-frame prediction mode is determined as the intra-frame prediction mode of the current block, or several finally selected intra-frame prediction modes are used as candidates for the current block's intra-frame prediction mode.
  • the higher the pattern in the MPM list the smaller the overhead in the code stream, which can also achieve the purpose of improving compression efficiency.
  • the method of template matching can be used to determine the two prediction modes of GPM. If the template matching method is used for GPM, one control switch can be used for the current block to control whether the two prediction modes of the current block use template matching, or two control switches can be used to control whether the two prediction modes use template matching respectively .
  • Another aspect is how to use template matching. For example, if GPM is used in merge mode, such as GPM in VVC, it uses merge_gpm_idxX to determine a motion information from mergeCandList, where the uppercase X is 0 or 1.
  • merge_gpm_idxX For the Xth motion information, one method is to use template matching method to optimize on the basis of the above motion information. That is, a motion information is determined from mergeCandList according to merge_gpm_idxX, and if template matching is used for the motion information, the template matching method is used to optimize on the basis of the above motion information.
  • Another method is not to use merge_gpm_idxX to determine a motion information from mergeCandList, but to search directly from a default motion information to determine a motion information.
  • the template matching method can be used to determine an intra-frame prediction mode, and there is no need to indicate the intra-frame prediction mode in the code stream index of. Or to determine a candidate set or MPM list by using a template matching method, it is necessary to indicate the index of the intra-frame prediction mode in the code stream.
  • the GPM can determine the area occupied by each prediction mode.
  • the so-called occupied area can be understood as the area whose weight corresponding to the prediction mode is the maximum value, or the area whose weight is greater than or equal to a certain threshold.
  • the reason why GPM can improve compression performance is because the two parts of the GPM "division" are different. Therefore, when using the template matching method to determine the prediction mode of the GPM, the template can also be divided.
  • the prior art can classify templates into 3 categories, namely left side, upper side and all (left side plus side).
  • the division of templates is related to the weight export mode. Exemplarily, as shown in Table 5, the template division in the prior art is related to the "division" angle or the "division" angle index angleIdx.
  • TM_A For example, record the left template as TM_A, the upper template as TM_L, and all (left plus side) templates as TM_AL.
  • Table 5 The relationship between the template and the "division" angle index is shown in Table 5. Some angle indexes such as 1, 6, and 7 are not used in the current GPM, so there is no corresponding template, which is represented by /.
  • the video decoding method provided by the embodiment of the present application will be introduced below with reference to FIG. 14 and by taking the decoding end as an example.
  • FIG. 14 is a schematic flowchart of a prediction method provided by an embodiment of the present application, and the embodiment of the present application is applied to the video decoder shown in FIG. 1 and FIG. 3 .
  • the method of the embodiment of the present application includes:
  • the encoding end needs to be in the code stream It is necessary to indicate the weight derivation mode, such as writing the index of the weight derivation mode in the code stream.
  • the weight derivation mode such as writing the index of the weight derivation mode in the code stream.
  • the decoding end derives the weight derivation mode through the prediction mode. Specifically, the decoder determines K initial prediction modes by decoding the code stream, and then determines the weight derivation mode of the current block according to the K initial prediction modes, and then determines the prediction value of the current block according to the weight derivation mode of the current block .
  • the above K initial prediction modes of the current block include the following examples:
  • Example 1 the above K initial prediction modes are all intra-frame prediction modes.
  • Example 2 the above K initial prediction modes are all inter-frame prediction modes.
  • Example 3 among the aforementioned K initial prediction modes, at least one is an intra-frame prediction mode, and at least one is an inter-frame prediction mode.
  • Example 4 among the above K initial prediction modes, at least one is an intra prediction mode, and at least one is a non-inter and non-intra prediction mode, for example, an intra-block copy IBC prediction mode or a palette prediction mode.
  • Example 5 among the aforementioned K initial prediction modes, at least one is an inter prediction mode, and at least one is a non-inter and non-intra prediction mode, for example, an IBC prediction mode or a palette prediction mode.
  • Example 6 none of the above K initial prediction modes is an intra prediction mode or an inter prediction mode, for example, one is an IBC prediction mode, one is a palette prediction mode, and so on.
  • Fig. 15 is a schematic diagram of using two prediction modes to predict the current block. As shown in Fig. 15, when predicting the current block, the first prediction mode can be used to determine the first prediction value, while the second prediction mode can be used to determine For the second predictive value, weighting may be performed on the first predictive value and the second predictive value by using weights to obtain the predictive value of the current block.
  • the decoder decodes the code stream and determines the K initial prediction modes, including but not limited to the following:
  • the code stream includes indexes of K initial prediction modes, so that the decoder can obtain the indexes of the K initial prediction modes by decoding the code stream, and then determine the K initial prediction modes according to the indexes.
  • the decoder decodes the code stream to obtain indexes 31 and 35, and then obtains the intra prediction modes corresponding to indexes 31 and 35 from the 65 angle prediction modes of VVC shown in Figure 10 as two initial prediction modes.
  • the above-mentioned S101 includes the steps from S101-A1 to S101-A3:
  • S101-A Determine a candidate prediction mode list, where the candidate prediction mode list includes at least two candidate prediction modes;
  • the candidate prediction modes included in the list of candidate prediction modes are all intra-frame prediction modes, for example including direct current (Direct Current, DC) at least one of a planar mode, a planar (PLANAR) mode, an angle mode, and the like.
  • the above candidate prediction mode list may also include intra prediction modes in the MPM list.
  • the candidate prediction modes included in the candidate prediction mode list are all inter-frame prediction modes, for example, include skip, merge, normal inter-frame At least one of prediction modes and the like.
  • the candidate prediction mode list includes at least one intra prediction mode and at least one inter prediction mode.
  • the above list of candidate prediction modes may also include modes such as IBC and palette.
  • the above list of candidate prediction modes may also include unidirectional prediction, bidirectional prediction, multi-hypothesis prediction, and the like.
  • the present application does not limit the types and numbers of the candidate prediction modes included in the above list of candidate prediction modes.
  • the embodiment of the present application does not limit the manner of determining the candidate prediction mode list.
  • the candidate prediction modes included in the above list of candidate prediction modes are preset modes.
  • the above list of candidate prediction modes is an MPM list.
  • the above list of candidate prediction modes is a set of prediction modes determined according to some rules, such as equidistant screening.
  • both the encoding end and the decoding end default that the candidate prediction modes in the above candidate prediction mode list are determined by template matching.
  • the decoder uses the prediction mode to predict the template of the current block to obtain the template According to the predicted value of the template and the reconstructed value of the template, determine the cost of the prediction mode, such as SAD cost, SATD cost or MSE cost. In this way, the decoder can sort the prediction modes according to the cost of each prediction mode to obtain a list of candidate prediction modes, for example, determine multiple prediction modes with the lowest cost as candidate prediction modes to form a list of candidate prediction modes.
  • the prediction mode index included in the code stream may be the index of at least one of the K initial prediction modes, for example, the prediction mode index is the index of the K initial prediction modes, or the K initial prediction mode index Index of the first initial prediction mode in the mode.
  • the decoding end decodes the code stream to obtain the prediction mode index, and determines K initial prediction modes from the determined candidate prediction mode list according to the prediction mode index.
  • the decoding end will select the K candidate prediction modes corresponding to the indexes of the K initial prediction modes in the candidate prediction mode list, Determined as K initial prediction modes.
  • the decoder determines the candidate prediction mode corresponding to the index in the candidate prediction mode list as For the first initial prediction mode in the K initial prediction modes, determine the K-1 candidate prediction modes corresponding to the K-1 indexes after the index in the candidate prediction mode list as the K initial prediction modes The remaining K-1 initial prediction modes.
  • the decoder determines K initial prediction modes according to the above steps, determines the weight derivation mode of the current block according to the K initial prediction modes, and then predicts the current block according to the weight derivation mode of the current block, Determine the predicted value for the current block.
  • the decoder before determining the K initial prediction modes, the decoder first needs to determine whether the current block uses K different prediction modes for weighted prediction processing. If the decoder determines that the current block uses K different prediction modes for weighted prediction processing, it executes the above S101 to determine K initial prediction modes.
  • the decoding end may determine whether the current block uses two different prediction modes for weighted prediction processing by determining the prediction mode parameter of the current block.
  • the prediction mode parameter may indicate whether the current block can use GPM mode or AWP mode, that is, indicate whether the current block can use K different prediction modes for prediction processing.
  • the prediction mode parameter can be understood as a flag indicating whether the GPM mode or the AWP mode is used.
  • the encoding end may use a variable as the prediction mode parameter, so that the setting of the prediction mode parameter may be realized by setting the value of the variable.
  • the encoder can set the value of the prediction mode parameter to indicate that the current block uses GPM mode or AWP mode.
  • the encoder can set the variable The value of is set to 1.
  • the encoding end can set the value of the prediction mode parameter to indicate that the current block does not use GPM mode or AWP mode, specifically, the encoding end can Set the variable value to 0. Furthermore, in the embodiment of the present application, after the encoding end completes the setting of the prediction mode parameters, it can write the prediction mode parameters into the code stream and transmit it to the decoding end, so that the decoding end can analyze the code stream Get the prediction mode parameters.
  • the decoder decodes the code stream to obtain the prediction mode parameters, and then determines whether the current block uses GPM mode or AWP mode according to the prediction mode parameters. If the current block uses GPM mode or AWP mode, it uses K different prediction modes. During prediction processing, K initial prediction modes are determined.
  • the GPM mode or the AWP mode is a prediction method, specifically, K different prediction modes are determined for the current block, and then determined according to the K different prediction modes. K predicted values, and then the weights can be determined again, and the K predicted values are combined according to the weights, and finally the predicted value of the current block can be obtained.
  • the size of the current block can be limited. It can be understood that since the prediction method proposed in the embodiment of the present application needs to use K different prediction modes to generate K prediction values, and then weight them according to the weight to obtain the prediction value of the current block, in order to reduce the complexity, the compression For the trade-off between performance and complexity, in the embodiment of the present application, the GPM mode or AWP mode may not be used for blocks of certain sizes. Therefore, in this application, the decoder can first determine the size parameter of the current block, and then determine whether the current block uses the GPM mode or the AWP mode according to the size parameter.
  • the size parameter of the current block may include the height and width of the current block. Therefore, the decoding end may use the height and width of the current block to restrict the use of GPM mode or AWP mode.
  • the decoding end may use the height and width of the current block to restrict the use of GPM mode or AWP mode.
  • the width is greater than the first threshold and the height is greater than the second threshold, it is determined that the current block uses the GPM mode or the AWP mode. It can be seen that a possible restriction is to use GPM mode or AWP mode only when the width of the block is greater than (or greater than or equal to) the first threshold and the height of the block is greater than (or greater than or equal to) the second threshold.
  • the values of the first threshold and the second threshold may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold.
  • the width is smaller than the third threshold and the height is larger than the fourth threshold, it is determined that the current block uses the GPM mode or the AWP mode. It can be seen that a possible restriction is to use the AWP mode only when the width of the block is smaller than (or smaller than or equal to) the third threshold and the height of the block is larger than (or larger than or equal to) the fourth threshold.
  • the values of the third threshold and the fourth threshold may be 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.
  • the limitation of the size of the block that can use the GPM mode or the AWP mode can also be realized through the limitation of the pixel parameters.
  • the decoding end may first determine the pixel parameters of the current block, and then further judge whether the current block can use the GPM mode or the AWP mode according to the pixel parameters and the fifth threshold. It can be seen that a possible restriction is to use the GPM mode or the AWP mode only when the pixel number of the block is greater than (or greater than or equal to) the fifth threshold. Wherein, the value of the fifth threshold may be 8, 16, 32 and so on. That is to say, in this application, the current block can use the GPM mode or the AWP mode only if the size parameter of the current block meets the size requirement.
  • this application there may be a frame-level flag to determine whether the current frame to be decoded uses the technical solution of this application. For example, it can be configured that an intra frame (such as an I frame) uses this application, and an inter frame (such as a B frame, P frame) does not use this application. Or it can be configured that intra frames do not use this application, and inter frames use this application. Or you can configure some inter frames to use this application and some inter frames not to use this application. Inter frames can also use intra prediction, thus inter frames are also likely to use this application.
  • an intra frame such as an I frame
  • an inter frame such as a B frame, P frame
  • a flag below the frame level and above the CU level such as tile, slice, patch, LCU, etc.
  • the weight derivation mode is used to determine the weight used by the current block.
  • the weight derivation mode may be a mode for deriving weights.
  • each weight export mode can export a weight matrix; for blocks of the same size, different weight export modes export different weight matrices.
  • AWP has 56 weight derivation modes
  • GPM has 64 weight derivation modes.
  • the decoding end determines the weight derivation mode of the current block according to the K initial prediction modes determined above, and in some cases, the encoding end may not write information related to the weight derivation mode of the current block in the code stream , thus saving codewords and reducing the encoding cost.
  • the embodiment of the present application does not limit the manner in which the decoding end determines the weight derivation mode of the current block according to the K initial prediction modes.
  • the decoding end determines the weight derivation mode of the current block according to the K initial prediction modes based on the template of the current block.
  • GPM has 64 weight derivation modes, and the decoder estimates which weight derivation modes are more likely to be selected and which weight derivation modes are less likely to be selected by analyzing the templates.
  • the decoder estimates which weight derivation modes are more likely to be selected and which weight derivation modes are less likely to be selected by analyzing the templates.
  • the GPM between frames a typical scene is that two objects move with each other, and GPM can be used for the edge of the two objects. Assuming that both the current block and the template contain the edge of the object, then the edge on the template can be used to infer the edge of the current block.
  • An example is as follows, given 2 motion information, according to the 2 motion information, the predicted values on the 2 templates are respectively obtained, which are recorded as the first template predicted value and the second template predicted value, and the template weight is derived according to a certain weight derivation mode , according to the first template prediction value, the second template prediction value and the template weight, determine the template prediction value corresponding to the weight derivation mode, because the reconstruction value of the template is available, so the template prediction value corresponding to the weight derivation mode and The reconstruction value of the template can get the cost of the weight derived mode on the template. This makes it possible to estimate the probability of each weight-derived mode being selected based on its cost on the template. One is to sort the weight-derived modes according to their cost on the template.
  • the mode with the least cost comes first, and the mode with the most cost follows. Because it is considered that the sorted weight derivation patterns are roughly arranged from high probability to low probability, so the method of variable length coding can be used for processing.
  • One possible implementation is to build a list of weight derivation modes, the encoder writes the weight derivation mode of the current block selected by the current block into the code stream according to the index in the list, and the decoder parses the index in the code stream, from The list determines the weight export mode for the current block.
  • Template matching uses the correlation between adjacent pixels to use some areas around the current block as templates. When encoding and decoding the current block, its left side and upper side have been decoded according to the encoding order. During inter-frame prediction, the best matching position of the template is found in the reference frame to determine the motion information or motion vector of the current block. During intra-frame prediction, a template is used to determine the intra-frame prediction mode of the current block.
  • the present application does not limit the specific shape of the template of the current block.
  • the template includes at least one of an upper decoded region, a left decoded region, and an upper left decoded region of the current block.
  • the width of the upper decoded area is the same as the width of the current block
  • the height of the left decoded area is the same as the height of the current block
  • the width of the upper left decoded area is the same as the width of the left decoded area
  • the upper left The height of the decoded region is the same as the height of the decoded region above.
  • the K prediction modes include the first prediction mode and the second prediction mode
  • the templates corresponding to the first prediction mode and the second prediction mode are the upper decoded blocks of the current block. area, or the decoded area on the left side of the current block, or the decoded area on the left side and the upper side of the current block.
  • the white area of the weight matrix of the current block is the weight corresponding to the predicted value of the first prediction mode
  • the black area is the weight corresponding to the predicted value of the second prediction mode. the weight of.
  • the first template corresponding to the first prediction mode is the upper decoded area of the current block
  • the second template corresponding to the second prediction mode is the left decoded area of the current block, however, the same as the second prediction mode
  • the adjacent template includes not only the left area, but also part of the upper decoded area. Therefore, the division of templates in the prior art is not fine enough, and the weight derivation mode of the current block cannot be accurately derived based on the template.
  • this application can realize a finer division of the template through the weight derivation mode. For example, as shown in FIG.
  • For division for example, record the template on one side of the dividing line as the first template, and record the template on the other side of the dividing line as the second template.
  • the first template corresponds to the first prediction mode
  • the second template corresponds to the second prediction mode
  • the weight of the first template is determined according to the weight corresponding to the first prediction mode
  • the weight of the second template is determined according to the weight corresponding to the second prediction mode
  • the weight of the first template is the same as the weight corresponding to the first prediction mode
  • the weight of the second template is the same as the weight corresponding to the second prediction mode.
  • the first template and the second template divided according to the above method may not be rectangular.
  • the first template and the second template have hypotenuses, and the cost calculation for irregular templates is more complicated.
  • the weight matrix can be directly extended to the template area, for example, extended to the left and upward, and the weight matrix can be overlaid on the template to determine the weight of the template.
  • the weight matrix can be directly extended to the template area, for example, extended to the left and upward, and the weight matrix can be overlaid on the template to determine the weight of the template.
  • the area in the inverted L-shaped frame on the left, upper left, and upper side is the template area, and the rectangular area at the lower right is the current block, so expand it to the upper left
  • the weight matrix of becomes the weight matrix of the template.
  • the weight derivation mode is used to simulate the template division, or the weight derivation mode is used to determine the template weight, and the template matching cost is determined according to the template weight. Since the division of templates may be different under different block shapes, this application does not need to set various rules for blocks of various shapes, for example, it is not necessary to set the correspondence between prediction modes and templates shown in Table 5. In terms of the accuracy of "division", the simulated division of the present application is also more precise and accurate than the method of only distinguishing the left area and the upper area.
  • the left area and the upper area of the current block shown in FIG. 17B may be used as the template of the current block, without considering the upper left matrix area.
  • Y can be used in YUV format because Y is dominant, and of course all components can be used.
  • Y can be used in YUV format because Y is dominant, and of course all components can be used.
  • template matching is performed on the Y component as an example, and reference can be made to other components.
  • deriving the template weight according to the weight derivation mode may be combined with the derivation process of the weight of the predicted value, for example, simultaneously deriving the weight of the template and the weight of the predicted value.
  • the weight of the predicted value can be understood as the weight corresponding to the predicted value, for example, use the first prediction mode to obtain the first predicted value, use the second prediction mode to obtain the second predicted value, and determine the first weight of the first predicted value according to the weight derivation mode , determine the second weight of the second predictor according to the weight derivation mode, and determine the sum of the product of the first predictor and the first weight and the product of the second predictor and the second weight as the predictor of the current block.
  • the first weight and the second weight above are referred to as the weight of the predicted value.
  • determining the template weight according to the weight derivation mode includes the following steps:
  • Step A determine the angle index and the distance index according to the weight derivation mode
  • Step B Determine the template weight according to the angle index, the distance index and the size of the template.
  • the present application may derive the template weight in the same manner as the weight of the predicted value.
  • an angle index and a distance index are determined according to the weight derivation mode, where the angle index can be understood as the boundary angle index of each weight derived by the weight derivation mode.
  • the angle index and distance index corresponding to the weight derivation mode can be determined according to the above Table 2. For example, if the weight derivation mode is 27, the corresponding angle index is 12 and the distance index is 3. Then, according to the angle index, the distance index and the size of the template, the template weight is determined.
  • the methods of exporting the template weight according to the weight export mode in the above step B include but are not limited to the following methods:
  • the first way is to directly determine the template weight according to the angle index, the distance index and the size of the template.
  • the above step B includes the following steps from step B11 to step B13:
  • Step B11 according to the angle index, the distance index and the size of the template, determine the first parameter of the pixel in the template;
  • Step B12 according to the first parameter of the pixel in the template, determine the weight of the pixel in the template
  • Step B13 Determine the template weight according to the weight of the pixel points in the template.
  • the weight of the pixel points in the template is determined according to the angle index, the distance index, the size of the template and the size of the current block, and then the weight matrix composed of the weights of each point in the template is determined as the template weight.
  • the first parameter of this application is used to determine the weights.
  • the first parameter is also referred to as a weight index.
  • the offset and the first parameter may be determined in the following manner:
  • the input of the template weight derivation process includes: as shown in Figure 18, the width of the current block nCbW, the height of the current block nCbH; the width of the left template nTmW, the height of the upper template nTmH; the "division" angle index variable angleId of GPM;
  • the distance index variable distanceIdx of the GPM; the component index variable cIdx, for example, the present application takes the luminance component as an example, so cIdx is 0, indicating the luminance component.
  • nW, nH, shift1, offset1, displacementX, displacementY, partFlip and shiftHor are derived as follows:
  • offsets offsetX and offsetY are derived as follows:
  • offsetY ((-nH)>>1)+(angleIdx ⁇ 16?(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3))
  • offsetX ((-nW)>>1)+(angleIdx ⁇ 16?(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)
  • the first parameter weightIdx is derived as follows:
  • weightIdx (((xL+offsetX) ⁇ 1)+1)*disLut[displacementX]+
  • the weight at the midpoint (x, y) of the template is determined according to the weightIdx.
  • the ways of determining the weight of the pixel in the template include but not limited to the following:
  • Mode 1 according to the first parameter of the pixel in the template, determine the second parameter of the pixel in the template; according to the second parameter of the pixel in the template, determine the weight of the pixel in the template.
  • the second parameter is also used to determine the weight.
  • the above-mentioned second parameter is also referred to as a weight index under the first component, and the first component may be a luma component, a chrominance component, and the like.
  • weightIdxL partFlip? 32+weightIdx:32-weightIdx
  • wTemplateValue[x][y] is the weight of the midpoint (x, y) of the template
  • weightIdxL is the second parameter of the midpoint (x, y) of the template, also known as the weight of the first component (such as the brightness component).
  • Weight index, wTemplateValue[x][y] is the weight of the template midpoint (x, y)
  • weightIdxL is 32–weightIdx.
  • weightIdxL is 32+weightIdx. It should be noted that 32 here is just a An example, the present application is not limited thereto.
  • the weight of the pixel in the template is determined.
  • the weight of the pixel point in the template is limited to the first preset value or the second preset value, that is to say, the weight of the pixel point in the template is either the first preset value value, or the second preset value, thereby reducing the computational complexity of the template weight.
  • the present application does not limit specific values of the first preset value and the second preset value.
  • the first preset value is 1.
  • the second preset value is 0.
  • the weight of the pixels in the template can be determined by the following formula:
  • wTemplateValue[x][y] is the weight of the midpoint (x, y) of the template, 1 in the above “1:0" is the first preset value, and 0 is the second preset value.
  • the weight of each point in the template is determined through the weight derivation mode, and the weight matrix composed of the weights of each point in the template is used as the template weight.
  • Method 2 Determine the weights of the current block and the template according to the weight derivation mode. That is to say, in this method 2, take the merged area composed of the current block and the template as a whole, and derive the weight of the pixels in the merged area according to the weight export mode. Weights. Based on this, the above step B includes the following steps B21 and S102-B22 steps:
  • Step B21 according to the angle index, the distance index, the size of the template and the size of the current block, determine the weight of the pixels in the merged area formed by the current block and the template;
  • Step B22 Determine the template weight according to the size of the template and the weight of the pixels in the merged area.
  • the current block and the template are taken as a whole, according to the angle index, the distance index, the size of the template and the size of the current block, the weight of the pixels in the merged area composed of the current block and the template is determined, and then according to the weight of the template Size, determine the weight corresponding to the template among the weights of the pixels in the merged area as the template weight, for example, as shown in Figure 18, determine the weight corresponding to the L-shaped template area in the merged area as the template weight.
  • the weight of the predicted value performs the subsequent forecasting process without re-determining the weight of the predicted value, thereby reducing the number of forecasting steps and improving the forecasting efficiency.
  • the present application does not limit the specific implementation manner of determining the weight of the pixels in the merged area formed by the current block and the template according to the angle index, the distance index, the size of the template, and the size of the current block.
  • the determination of the weights of the pixels in the merged area in the above step B21 includes the following steps from step B211 to step B212:
  • Step B211 according to the angle index, the distance index and the size of the merged area, determine the first parameter of the pixel in the merged area;
  • Step B212 according to the first parameter of the pixels in the merged area, determine the weight of the pixels in the merged area.
  • the weight of the pixel points in the merged area is determined, and the weight of each point in the merged area forms a weight matrix.
  • the offset and the first parameter may be determined in the following manner:
  • the inputs to the weight derivation process of the merged area are: as shown in Figure 18, the width of the current block nCbW, the height of the current block nCbH; the width of the left template nTmW, the height of the upper template nTmH; the "division" angle index variable angleId of the GPM ;
  • the distance index variable distanceIdx of GPM; the component index variable cIdx, for example, this application takes the brightness component as an example, so cIdx is 0, indicating the brightness component.
  • nW, nH, shift1, offset1, displacementX, displacementY, partFlip and shiftHor are derived as follows:
  • offsets offsetX and offsetY are derived as follows:
  • offsetY ((-nH)>>1)+(angleIdx ⁇ 16?(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3))
  • offsetX ((-nW)>>1)+(angleIdx ⁇ 16?(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)
  • disLut is determined according to Table 3.
  • the first parameter weightIdx is derived as follows:
  • weightIdx (((xL+offsetX) ⁇ 1)+1)*disLut[displacementX]+
  • the weight at the middle point (x, y) of the merged area is determined according to the weightIdx.
  • the ways of determining the weight of the pixels in the merged region include but not limited to the following:
  • Method 1 according to the first parameter of the pixels in the merged area, determine the second parameter of the pixels in the merged area; according to the second parameter of the pixels in the merged area, determine the weight of the pixels in the merged area.
  • weightIdx (((xL+offsetX) ⁇ 1)+1)*disLut[displacementX]+
  • weightIdxL partFlip? 32+weightIdx:32-weightIdx
  • wValueMatrix[x][y] is the weight of the midpoint (x, y) of the merged area
  • weightIdxL is the second parameter of the midpoint (x, y) of the merged area
  • wTemplateValue[x][y] is the midpoint of the merged area (x, y) weights.
  • the weight of the pixels in the merged area is determined.
  • the weight of the pixels in the merged area is the first preset value or the second preset value.
  • the weight of the pixel in the merged area is limited to the first preset value or the second preset value, that is to say, the weight of the pixel in the merged area is either the first preset value or the second preset value.
  • a preset value, or a second preset value so as to reduce the computational complexity of merging region weights.
  • the present application does not limit specific values of the first preset value and the second preset value.
  • the first preset value is 1.
  • the second preset value is 0.
  • the weight of the pixels in the merged area can be determined by the following formula:
  • wValueMatrix[x][y] is the weight of the point (x, y) in the merged region, 1 in the above “1:0" is the first preset value, and 0 is the second preset value.
  • the weight of each point in the merged area composed of the current block and the template is determined through the weight derivation mode, and then according to the size of the template, the weight corresponding to the template in the merged area is determined as the template weight, and the method 2
  • the weight of the predicted value is determined at the same time, and there is no need to perform subsequent steps of determining the weight of the predicted value, which reduces the number of prediction steps and improves the prediction efficiency.
  • the decoding end derives the mode according to the weight of the current block, and the ways of determining the prediction value of the current block include but are not limited to the following:
  • the decoder determines the predicted value of the current block according to the weight derivation mode of the current block and K initial prediction modes, specifically, the decoder determines the weight of the prediction value according to the weight derivation mode of the current block, and then according to the K initial prediction mode to predict the current block to obtain K predicted values, and then use the weight of the predicted values to perform weighted average calculation on the K predicted values to obtain the predicted value of the current block.
  • the weight of the prediction value derived according to the weight derivation mode of the current block can be understood as deriving the weight corresponding to each pixel in the current block, or can be understood as deriving the weight matrix corresponding to the current block.
  • K predicted values corresponding to each pixel in the current block may be determined, and according to the K predicted values corresponding to each pixel and the weight of the predicted value, A prediction value corresponding to each pixel is determined, and the prediction value corresponding to each pixel in the current block constitutes a prediction value of the current block.
  • determining the predicted value of the current block based on the weight of the predicted value it can also be performed according to the block, for example, determine K predicted values of the current block, and calculate the K predicted values of the current block according to the weight matrix of the current block Weighted to get the predicted value of the current block.
  • the decoder can use the weight derivation mode of the current block to improve the above K initial prediction modes.
  • the above S103 includes the following steps of S103-A and S103-B:
  • S103-B Determine the prediction value of the current block according to the K adjusted prediction modes and the weight derivation mode of the current block.
  • the GPM has K+1 elements, that is, one weight derivation mode and K prediction modes. Since one weight derivation mode and K prediction modes jointly generate a prediction block, this prediction block acts on the current block. They are related. To give a few examples, say the current block contains the edges of 2 relatively moving objects, which is an ideal scenario for inter-frame GPM. In theory, this "division" should occur at the edge of the object, but in fact there are limited possibilities for "division", and it is impossible to cover any edge. Sometimes a similar "division" is selected, so there may be more than one similar "division". One, which one to choose depends on which "division" and the combination of the two prediction modes have the best results.
  • the weight derivation mode can be used to determine the prediction mode, and the prediction mode can also be used to determine the weight derivation mode.
  • some schema information can also be refined.
  • the so-called improvement is to obtain a better model information by collecting more relevant information and using more calculations on the basis of an initial model information.
  • refine is the operation of the decoder.
  • the motion information can also be improved and refined based on the template. That is, on the basis of the initial motion information, a small-scale search is performed at the decoding end, and the motion information is optimized according to the degree of matching between the searched motion information and the template.
  • One understanding of the so-called matching degree between motion information and the template is the matching degree between the predicted value of the motion information on the template and the reconstructed value of the template.
  • the weight derivation mode can be determined according to the prediction mode, and then refine the prediction mode according to the weight derivation mode. Or first determine the prediction mode according to the weight derivation mode, and then return to derive the mode according to the prediction mode refine weight.
  • the two can also be implemented alternately, if the complexity allows.
  • Templates can be used as a link to implement these methods, and further, template weights can be used as a link to implement these methods. That is, the cost on the template is calculated according to the predicted value of the prediction mode on the template and the sub-template or template weight corresponding to the weight derivation mode. The cost on the template is used to derive or determine the prediction mode or weight derivation mode.
  • the weight derivation mode of the current block is determined according to the K initial prediction modes, and then the K initial prediction modes are improved by using the weight derivation mode of the current block to obtain K adjusted prediction modes , and then determine the prediction value of the current block according to the weight derivation mode of the current block and the K adjusted prediction modes.
  • the above S102 includes the following steps from S102-A to S102-C:
  • i is a positive integer from 1 to N;
  • the decoding end first determines N candidate weight derivation modes, and the N candidate weight derivation modes form a list of weight derivation modes, and the decoding end calculates the corresponding The weight derivation mode of the current block is determined from the N candidate weight derivation modes according to the first cost.
  • the candidate weight derivation mode with the smallest cost among the N candidate weight derivation modes is determined as the weight derivation mode of the current block.
  • sort the N candidate weight derivation modes to obtain a list of candidate weight derivation modes; decode the code stream to obtain a weight derivation mode index; export the candidate weight derivation mode list
  • the middle weight derivation mode index corresponds to the candidate weight derivation mode, and is determined as the weight derivation mode of the current block.
  • the foregoing first cost may be a cost such as SAD, SATD, or SEE.
  • the process for the decoder to determine the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes is the same. Take the price as an example.
  • the ways of determining the first cost corresponding to the i-th candidate weight derivation mode include but are not limited to the following:
  • Method 1 Use the i-th candidate weight derivation mode and K initial prediction modes to predict the template of the current block to obtain the first predicted value of the template; determine the i-th The first cost corresponding to the weight export mode.
  • the template weight is determined according to the i-th candidate weight derivation mode, and K initial prediction modes are used to predict the template to obtain K predicted values of the template, and the template weight is used to weight the K predicted values corresponding to the template to obtain
  • the predicted value of the template for the convenience of description, the predicted value of the template is recorded as the first predicted value of the template.
  • the prediction cost of the i-th candidate weight derivation mode with respect to the template is determined, and the prediction cost is recorded as the first cost corresponding to the i-th weight derivation mode.
  • the specific process of determining the template weight according to the i-th candidate weight derivation mode can refer to the above-mentioned specific content description of determining the template weight according to the weight derivation mode, which will not be repeated here.
  • the first cost corresponding to each candidate weight derivation mode among the N candidate weight derivation modes can be determined, and then the candidate weight derivation mode with the smallest first cost among the N candidate weight derivation modes is determined as the weight of the current block Derivation mode, or sort the N candidate weight derivation modes according to the first costs corresponding to the N candidate weight derivation modes to obtain a list of candidate weight derivation modes, and index the weight derivation mode in the candidate weight derivation mode list corresponding to the candidate weight Export mode, determined as the weight export mode of the current block.
  • determining K adjusted prediction modes includes the following steps:
  • S103-A1 includes but is not limited to the following examples:
  • Example 1 Determine at least one prediction mode related to the weight derivation mode of the current block.
  • the at least one prediction mode may be a prediction mode whose prediction angle is parallel or approximately parallel to the boundary line of the weight derivation mode of the current block or a prediction mode whose prediction angle is parallel to the current block’s weight derivation mode.
  • the weights of the blocks are derived from the mode's vertical or nearly vertical prediction modes.
  • the combination includes K prediction modes.
  • the K prediction modes included in the new prediction mode combination and the weight derivation mode of the current block For each new prediction mode combination, use the K prediction modes included in the new prediction mode combination and the weight derivation mode of the current block to predict the target, and obtain the predicted value of the template. According to The predicted value of the template and the reconstructed value of the template determine the cost corresponding to the new prediction mode combination, and then the cost corresponding to each new prediction mode combination in the plurality of new prediction mode combinations formed above can be obtained. Similarly, the weight derivation mode of the current block and the K initial prediction modes are used to predict the current block, and the costs corresponding to the K initial prediction modes are determined. In this way, the decoder can obtain K adjusted prediction modes according to the cost corresponding to the new prediction mode combination and the costs corresponding to the K initial prediction modes.
  • the new The K prediction modes included in the prediction mode combination 1 are determined as K adjusted prediction modes, and the new prediction mode combination 1 may include at least one initial prediction mode, or may not include the initial prediction mode.
  • Example 2 through the following steps from S103-A11 to S103-A14, K initial prediction modes are improved to obtain K adjusted prediction modes:
  • the specific process of determining the weight of the first template can refer to the specific description of the above embodiment of deriving the weight of the template, which will not be repeated here.
  • the weight derivation mode in the above template weight derivation embodiment is the weight derivation mode of the current block, and the corresponding template weight is the first template weight.
  • the above S103-A12 and the above S103-A11 are executed in no order, that is, the above S103-A12 can be executed after the above S103-A11, or can be executed before the above S103-A11, or it can be executed with the above S103-A11.
  • A11 is executed synchronously, which is not limited in this embodiment of the present application.
  • the decoding end uses the weight derivation mode of the current block to improve each of the K initial prediction modes, and obtains the adjusted prediction mode corresponding to each of the K initial prediction modes, and then obtains K adjusted prediction modes.
  • the process of determining the adjusted prediction mode corresponding to each of the K initial prediction modes is the same.
  • the adjusted Forecast mode is used as an example.
  • the decoder first determines M first candidate prediction modes corresponding to the jth initial prediction mode.
  • the manner in which the decoding end determines the M first candidate prediction modes corresponding to the jth initial prediction mode includes the following two situations:
  • M intra-frame prediction modes similar to the j-th initial prediction mode are determined as M first candidate prediction modes. For example, if the jth initial prediction mode is an angle prediction mode, M angle prediction modes that are similar (or similar) to the angle of the jth initial prediction mode are used to determine the M first candidates corresponding to the jth initial prediction mode predictive mode.
  • the j-th initial prediction mode is an inter-frame prediction mode
  • search according to the j-th initial prediction mode to obtain M pieces of first motion information
  • Corresponding M first candidate prediction modes For example, for the jth initial prediction mode, its motion vector is (xInit, yInit), set a search range such as a rectangular area from xInit-sR to xInit+sR in the horizontal direction, and from yInit-sR to yInit+sR in the vertical direction , where sR can be 2, 4, 8, etc.
  • Each motion vector in the rectangular area can be combined with other information of the jth initial prediction mode, such as reference frame index and prediction list flag, etc. to determine a motion information, thereby determining a prediction mode, so that the jth can be obtained M first candidate prediction modes corresponding to the prediction modes.
  • other information of the jth initial prediction mode such as reference frame index and prediction list flag, etc.
  • one way of obtaining M pieces of first motion information by searching according to the jth initial prediction mode is to search within a preset search range according to the motion vector corresponding to the jth initial prediction mode , to obtain M offsets, and each offset includes an offset in the first direction (for example, the x direction) and an offset in the second direction (for example, the y direction). Furthermore, according to the M offsets, M pieces of first motion information are obtained, for example, for each of the M offsets, the motion vector corresponding to the jth initial prediction mode is calculated according to the offset Offset is performed to obtain first motion information.
  • the first candidate prediction mode uses the first candidate prediction mode to predict the template of the current block to obtain a prediction value, and then, according to the first template weight and The prediction value determines a second cost of the first candidate prediction mode with respect to the template.
  • the calculation of the second cost of a first candidate prediction mode with respect to a template among the M first candidate prediction modes is taken as an example for illustration below.
  • the i-th pixel in the template can be understood as any point in the template, that is, the process of determining the second cost of each point in the template is the same, Just refer to the i-th pixel.
  • the first candidate prediction mode is used to predict the template to obtain the predicted value of the template, and the predicted value corresponding to the i-th pixel in the predicted value of the template is recorded as the i-th predicted value, and the i-th
  • the reconstruction value corresponding to the pixel in the reconstruction value of the template is recorded as the i-th reconstruction value, and then according to the i-th predicted value and the i-th reconstruction value, and the weight of the i-th pixel in the first template weight, determine The second cost of the first candidate prediction mode at the i-th pixel.
  • the second cost of the first candidate prediction mode at each point or multiple points in the template is determined, and then the first candidate prediction mode is determined according to the second cost of each point or multiple points in the template.
  • the second cost of the prediction model with respect to the template For example, the sum of the second costs of the first candidate prediction mode at the pixel points in the template is determined as the second cost of the first candidate prediction mode with respect to the template.
  • the second cost of the first candidate prediction mode at the ith pixel point (x, y) in the template can be determined according to the following formula (1):
  • tempValueA[x][y] abs(predTemplateSamplesCandA[x][y]-recTemplateSamples[x][y])*wTemplateValue[x][y]1 (1)
  • the second cost of the first candidate prediction mode with respect to the template is determined according to the following formula (2):
  • abs(predTemplateSamplesCandA[x][y]-recTemplateSamples[x][y]) is the absolute value of the difference between the predicted value predTemplateSamplesCandA and the reconstructed value recTemplateSamples of the template midpoint (x, y), and the absolute value of the difference is called is the second cost corresponding to point (x, y). Multiply this value by the weight value wTemplateValue[x][y] of the point in the first template weight to obtain tempValueA[x][y]. tempValueA[x][y] can be regarded as the second cost of the first candidate prediction mode at this point (x, y).
  • the total second cost costCandA of the first candidate prediction mode on the template is the accumulation of the second cost of each point on the template.
  • the above takes SAD as an example to determine the second cost of the first candidate prediction mode with respect to the template.
  • the second cost of the first candidate prediction mode with respect to the template can also be determined according to cost calculation methods such as SATD and MSE .
  • the second cost corresponding to each first candidate prediction mode among the M first candidate prediction modes can be determined, and then, the following S103-A14 is performed.
  • the first candidate prediction mode with the smallest second cost among the M first candidate prediction modes is determined as the adjusted prediction mode corresponding to the jth initial prediction mode.
  • the above embodiment is described by taking determining the adjusted prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes as an example. Referring to the above description, the adjustment corresponding to each initial prediction mode among the K initial prediction modes can be determined. prediction mode, and then get K adjusted prediction modes.
  • the decoder first determines the weight derivation mode of the current block according to the K initial prediction modes, and then improves the K initial prediction modes according to the weight derivation mode of the current block, for example, through the above example 1 or example 2
  • the method is improved to obtain K adjusted prediction modes, and then the current block is predicted by using the weight derivation mode of the current block and the K adjusted prediction modes to obtain the prediction value of the current block.
  • the decoder in addition to using the method 1 above to determine the weight derivation mode of the current block and the K adjusted prediction modes, the decoder can also determine the weight derivation mode of the current block and the K adjusted prediction modes through the following method 2 predictive mode.
  • the second way is to integrate the improvement of the prediction mode and the determination of the weight derivation mode of the current block together, and carry out synchronously.
  • determine the i-th The first cost corresponding to the candidate weight derivation mode includes the following steps from S102-B11 to S102-B12:
  • the second method for the i-th candidate weight derivation mode among the N candidate prediction modes, first use the i-th candidate weight derivation mode to improve the K initial prediction modes to obtain the i-th candidate weight derivation mode
  • the corresponding K improved prediction modes for the convenience of description, the K improved prediction modes corresponding to the i-th candidate weight derivation mode are recorded as K first improved prediction modes.
  • the i-th candidate weight derivation mode, and the K first improved prediction modes corresponding to the i-th candidate weight derivation mode determine the first cost corresponding to the i-th candidate weight derivation mode, so that a more accurate current
  • the weight of the block derives the mode and K adjusted prediction modes.
  • S102-B11 includes but is not limited to the following examples:
  • the at least one prediction mode may be a prediction mode whose prediction angle is parallel or approximately parallel to the boundary line of the i-th candidate weight derivation mode or a prediction angle A prediction mode that is vertical or approximately perpendicular to the boundary of the i-th candidate weight derivation mode.
  • each new prediction The mode combination includes K prediction modes, and for each new prediction mode combination, use the K prediction modes included in the new prediction mode combination and the i-th candidate weight derivation mode to predict the template, and obtain the predicted value of the template , according to the predicted value of the template and the reconstruction value of the template, determine the cost corresponding to the new prediction mode combination, and then obtain the cost corresponding to each new prediction mode combination in the plurality of new prediction mode combinations formed above.
  • the decoder can obtain K first improved prediction modes according to the cost corresponding to the new prediction mode combination and the costs corresponding to the K initial prediction modes. For example, if the cost corresponding to the new prediction mode combination 1 is the smallest, then the The K prediction modes included in the new prediction mode combination 1 are determined as K first improved prediction modes, and the new prediction mode combination 1 may include at least one initial prediction mode, or may not include the initial prediction mode.
  • Example 2 through the following steps from S102-B11-1 to S102-B11-4, K initial prediction modes are improved to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode:
  • the specific process of determining the second template weight can refer to the specific description of the above template weight derivation embodiment, which will not be repeated here.
  • the weight derivation mode is the ith candidate weight derivation mode
  • the corresponding template weight is the second template weight.
  • M is a positive integer
  • j is a number from 1 to K positive integer.
  • the above S102-B11-2 can be executed after the above S102-B11-1, or after S102-B11 -1 is executed before, or executed synchronously with the above S102-B11-1, which is not limited in this embodiment of the present application.
  • the decoder uses the i-th candidate weight derivation mode to improve each of the K initial prediction modes, and obtains each of the K initial prediction modes under the i-th candidate weight derivation mode.
  • the first improved prediction mode corresponding to the mode, and then K first improved prediction modes are obtained.
  • the process of determining the first improved prediction mode corresponding to each initial prediction mode in the K initial prediction modes is the same.
  • An example of improving the prediction model will be described.
  • the decoder first determines M first candidate prediction modes corresponding to the jth initial prediction mode.
  • the manner in which the decoding end determines the M first candidate prediction modes corresponding to the jth initial prediction mode includes the following two situations:
  • one way of obtaining M pieces of first motion information by searching according to the jth initial prediction mode is to search within a preset search range according to the motion vector corresponding to the jth initial prediction mode , to obtain M offsets, and each offset includes an offset in the first direction (for example, the x direction) and an offset in the second direction (for example, the y direction). Furthermore, according to the M offsets, M pieces of first motion information are obtained, for example, for each of the M offsets, the motion vector corresponding to the jth initial prediction mode is calculated according to the offset Offset is performed to obtain first motion information.
  • the first candidate prediction mode uses the first candidate prediction mode to predict the template of the current block to obtain a prediction value, and then, according to the second template weight and The prediction value determines a third cost of the first candidate prediction mode with respect to the template.
  • the following takes the calculation of the third cost of a first candidate prediction mode with respect to a template among the M first candidate prediction modes as an example for illustration.
  • the i-th pixel in the template can be understood as any point in the template, that is, the process of determining the third cost of each point in the template is the same, Just refer to the i-th pixel.
  • the first candidate prediction mode is used to predict the template to obtain the predicted value of the template, and the predicted value corresponding to the i-th pixel in the predicted value of the template is recorded as the i-th predicted value, and the i-th
  • the reconstruction value corresponding to the pixel point in the reconstruction value of the template is recorded as the i-th reconstruction value, and then according to the i-th predicted value and the i-th reconstruction value, and the weight of the i-th pixel point in the second template weight, determine The third cost of the first candidate prediction mode at the i-th pixel.
  • the third cost of the first candidate prediction mode at each point or multiple points in the template is determined, and then the first candidate prediction mode is determined according to the third cost of each point or multiple points in the template.
  • the third cost of the prediction model with respect to the template For example, the sum of the third costs of the first candidate prediction mode at the pixel points in the template is determined as the third cost of the first candidate prediction mode with respect to the template.
  • the third cost of the first candidate prediction mode with respect to the template can be determined by referring to the above formulas (1) and (2), specifically, replacing wTemplateValue[x][y] in the above formula (1) with pixels The weight value corresponding to the point (x, y) on the second template weight.
  • the third cost corresponding to each first candidate prediction mode among the M first candidate prediction modes can be determined, and then, the following S102-B11-4 is performed.
  • the first candidate prediction mode with the smallest third cost among the M first candidate prediction modes is determined as the first improved prediction mode corresponding to the jth initial prediction mode.
  • the above embodiment is described by taking the determination of the first improved prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes as an example. Referring to the above description, it is possible to determine the first improved prediction mode corresponding to each of the K initial prediction modes. The first improved prediction mode, and then K first improved prediction modes under the i-th candidate weight derivation mode are obtained.
  • use the i-th candidate weight derivation mode to derive the second template weight use K first improved prediction modes to predict the template, obtain K predicted values, and use the second template weight to weight the K predicted values, A second predicted value of the template is obtained, and a first cost corresponding to the i-th weight derivation mode is determined according to the second predicted value of the template and the reconstructed value of the template.
  • the determination of the first cost corresponding to the i-th candidate weight derivation mode is taken as an example.
  • the above-mentioned determination of the first cost corresponding to the i-th weight derivation mode can be used in this way, the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes is determined, and then the weight derivation mode of the current block is determined according to the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes. model.
  • the candidate weight derivation mode with the smallest cost among the N candidate weight derivation modes is determined as the weight derivation mode of the current block.
  • sort the N candidate weight derivation modes to obtain a list of candidate weight derivation modes; decode the code stream to obtain a weight derivation mode index; export the candidate weight derivation mode list
  • the middle weight derivation mode index corresponds to the candidate weight derivation mode, and is determined as the weight derivation mode of the current block.
  • K first improved prediction modes corresponding to each candidate weight derivation mode are determined.
  • the derivation mode according to the weight of the current block in the above S103-A includes the following steps:
  • each candidate weight derivation mode is determined Corresponding K first improved prediction modes.
  • K first improved prediction modes corresponding to the weight derivation mode of the current block can be determined as K adjusted prediction modes, and then the weight of the current block can be used
  • the derived mode and the K adjusted prediction modes predict the current block to obtain the predicted value of the current block.
  • K first improved prediction modes corresponding to each candidate weight derivation mode are determined, and then according to each candidate weight derivation mode and its For the K first improved prediction modes, the template is predicted, and the first cost is determined. Based on the first cost, the weight derivation mode of the current block can be more accurately determined from the N candidate weight derivation modes.
  • the decoder in addition to using the method 1 or method 2 above to determine the weight derivation mode of the current block and the K adjusted prediction modes, can also determine the weight derivation mode and K of the current block through the following method 3 an adjusted forecast model.
  • each candidate weight derivation mode is not used to improve the initial prediction mode, but simple P sub-templates are used, such as the current All templates, left templates, and upper templates of the block improve the initial prediction mode, thereby reducing the amount of calculation for improving the initial prediction mode.
  • determining the first cost corresponding to the i-th candidate weight derivation mode includes the following steps from S102-B21 to S102-B23:
  • the decoding end uses P sub-templates to improve K initial prediction modes, and obtains a second improved prediction mode for each of the K initial prediction modes with respect to the P sub-templates.
  • the second improvement of each initial prediction mode from the K initial prediction modes with respect to the P sub-templates K second improved prediction modes corresponding to the candidate weight derivation mode are obtained, and then the template of the current block is predicted according to the candidate weight derivation mode and its corresponding K second prediction modes, and the candidate weight derivation mode is determined.
  • the first cost corresponding to the mode is obtained.
  • the weight derivation mode of the current block is determined according to the first cost corresponding to each weight derivation mode in the N candidate weight derivation modes.
  • the candidate prediction modes are no longer used alone to improve the K initial prediction modes, thereby reducing the complexity of improving the initial prediction modes.
  • the P sub-templates include at least one of the left template TM_L, the upper template TM_A and the entire template TM_AL of the current block, wherein the entire template TM_AL includes the left template TM_L and the upper template TM_A.
  • the process of determining each of the K initial prediction modes at the decoding end is the same as the second improved prediction mode of the P sub-templates.
  • the j-th initial prediction mode is described with regard to the second improved prediction mode of the P sub-templates as an example.
  • determining the second improved prediction mode of the jth initial prediction mode with respect to the P sub-templates includes: determining the first Determine the second improved prediction mode of the j initial prediction mode with respect to the left template TM_L, determine the second improved prediction mode of the jth initial prediction mode with respect to the upper template TM_A, and determine the jth initial prediction mode with respect to the entire template TM_AL 2. Improve the prediction mode. That is to say, in this step, the decoding end determines the second improved prediction mode of the j-th initial prediction mode for each sub-template in the P sub-templates.
  • the embodiment of the present application does not limit the specific manner of determining the second improved prediction mode of the j-th initial prediction mode with respect to the P sub-templates.
  • S102-B21 includes the following steps from S102-B21-1 to S102-B21-3:
  • the way for the decoding end to determine the M first candidate prediction modes corresponding to the jth initial prediction mode includes the following two situations:
  • searching according to the j th initial prediction mode to obtain M pieces of first motion information includes: searching according to the i th initial prediction mode to obtain M offsets; according to the M offsets amount to obtain M pieces of first motion information.
  • p is a positive value from 1 to P integer.
  • the decoding end After determining the M first candidate prediction modes corresponding to the jth initial prediction mode according to the method of the above case 1 or case 2, the decoding end determines that the M first candidate prediction modes are respectively related to each sub-template in the P sub-templates The corresponding cost, and then determine the second improved prediction mode corresponding to each sub-template of the j-th initial prediction mode with respect to each of the P sub-templates according to the cost. For ease of description, the cost corresponding to the first candidate prediction mode at this time is recorded as the fourth cost.
  • the fourth cost of the first candidate prediction mode with respect to the P th sub template can be determined.
  • the jth initial prediction mode corresponds to three first candidate prediction modes, which are respectively denoted as the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3, assuming that the P sub-templates include the left Template TM_L, upper template TM_A, and all templates TM_AL.
  • the left template TM_L use the first candidate prediction mode 1, the first candidate prediction mode 2 and the first candidate prediction mode 3 to predict the left template TM_L respectively, and the corresponding prediction values are prediction value 1, prediction Value 2 and predicted value 3.
  • the fourth cost of the first candidate prediction mode 1 with respect to the left template TM_L is determined, and according to the prediction value 2 and the reconstruction value of the left template TM_L, the first The fourth cost of the candidate prediction mode 2 with respect to the left template TM_L determines the fourth cost of the first candidate prediction mode 3 with respect to the left template TM_L according to the prediction value 3 and the reconstructed value of the left template TM_L.
  • the upper template TM_A use the first candidate prediction mode 1, the first candidate prediction mode 2 and the first candidate prediction mode 3 to predict the upper template TM_A respectively, and the corresponding predicted values are predicted values 4, Predicted value 5 and predicted value 6.
  • the fourth cost of the first candidate prediction mode 1 with respect to the upper template TM_A is determined, and according to the predicted value 5 and the reconstruction value of the upper template TM_A, the first The fourth cost of the upper template TM_A for the candidate prediction mode 2 determines the fourth cost of the upper template TM_A for the first candidate prediction mode 3 according to the prediction value 6 and the reconstructed value of the upper template TM_A.
  • the fourth cost of each first candidate prediction mode with respect to each of the P sub-templates is determined, and then, the following S102-B21 is performed -3.
  • the first candidate prediction mode corresponding to the smallest fourth cost on the p-th sub-template is determined as the second improved prediction mode of the j-th initial prediction mode on the p-th sub-template.
  • the j-th initial prediction mode corresponds to three first candidate prediction modes, which are respectively denoted as the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3, assuming that the P sub-templates include Left template TM_L, upper template TM_A, and all templates TM_AL.
  • the fourth cost of the first candidate prediction mode 1 with respect to the upper template TM_A is less than the fourth cost of the first candidate prediction mode 2 with respect to the upper template TM_A
  • the first candidate prediction mode 2 with respect to the upper template TM_A The fourth cost of is less than the fourth cost of the first candidate prediction mode 3 with respect to the upper template TM_A, so that the first candidate prediction mode 1 can be determined as the second improved prediction mode of the jth initial prediction mode with respect to the left template TM_L.
  • the fourth cost of the first candidate prediction mode 2 with respect to the upper template TM_A is smaller than the fourth cost of the first candidate prediction mode 1 with respect to the upper template TM_A
  • the first candidate prediction mode 1 with respect to the upper template TM_A The fourth cost of the side template TM_A is smaller than the fourth cost of the first candidate prediction mode 3 with respect to the upper template TM_A, so that the first candidate prediction mode 2 can be determined as the second improvement of the jth initial prediction mode with respect to the upper template TM_A predictive mode.
  • the fourth cost of the first candidate prediction mode 2 for all templates TM_AL is less than the fourth cost of the first candidate prediction mode 1 for all templates TM_AL
  • the first candidate prediction mode 1 for all templates TM_AL The fourth cost is smaller than the fourth cost of the first candidate prediction mode 3 with respect to all templates TM_AL, so that the first candidate prediction mode 2 can be determined as the second improved prediction mode of the jth initial prediction mode with respect to all templates TM_AL.
  • the second improved prediction mode of each initial prediction mode for each sub-template in the P sub-templates can be determined.
  • each initial prediction mode from the K initial prediction modes is related to the P sub-templates Among the second improved prediction modes of the template, K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined.
  • the sub-templates corresponding to the i-th candidate weight derivation mode are the upper template TM_A and all templates TM_AL.
  • the improved prediction mode is the second improved prediction mode 12
  • the second improved prediction mode of the second prediction mode for all templates TM_AL is the second improved prediction mode 23
  • two second second prediction modes corresponding to the i-th candidate weight derivation mode can be determined.
  • the improved prediction modes are respectively the second improved prediction mode 12 and the second improved prediction mode 23 .
  • the decoding end determines the division angle corresponding to the i-th candidate weight derivation mode, and according to the correspondence between the division angle and the template of the prediction mode (refer to the above table 5 for details), from the K Among the second improved prediction modes of the P sub-templates in the initial prediction mode, K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined.
  • K second improved prediction modes corresponding to the i-th candidate weight mode are determined, and then the following steps of S102-B23 are performed.
  • the first cost corresponding to the i weight derivation mode For example, determine the template weight according to the i-th candidate weight derivation mode, use K second prediction improvement modes to predict the template of the current block respectively, obtain K predicted values, and use the determined template weight to weight the K predicted values processing to obtain the template prediction value corresponding to the i-th candidate weight derivation mode, and then determine the first cost corresponding to the i-th candidate weight derivation mode according to the template prediction value and the template reconstruction value.
  • the determination of the first cost corresponding to the i-th candidate weight derivation mode is taken as an example.
  • the above-mentioned determination of the first cost corresponding to the i-th weight derivation mode can be used in this way, the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes is determined, and then the weight derivation mode of the current block is determined according to the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes. model.
  • the candidate weight derivation mode with the smallest cost among the N candidate weight derivation modes is determined as the weight derivation mode of the current block.
  • sort the N candidate weight derivation modes to obtain a list of candidate weight derivation modes; decode the code stream to obtain a weight derivation mode index; export the candidate weight derivation mode list
  • the middle weight derivation mode index corresponds to the candidate weight derivation mode, and is determined as the weight derivation mode of the current block.
  • P sub-templates are used to improve the K initial prediction modes to obtain a second improved prediction mode for each of the P sub-templates for each of the K initial prediction modes.
  • the K initial prediction modes are no longer improved according to different candidate weight derivation modes, but are determined uniformly from the above
  • the K initial prediction modes of each initial prediction mode in the second improved prediction mode of each sub-template in each of the P sub-templates, the K second improved prediction modes corresponding to each candidate weight derivation mode are queried, thereby reducing the need for the initial
  • the complexity of the prediction mode is improved, and the speed of calculating the first cost corresponding to each candidate weight derivation mode is increased, so that the weight derivation mode of the current block can be quickly determined from the N candidate weight derivation modes.
  • K second improved prediction modes corresponding to each candidate weight derivation mode among the N candidate weight derivation modes are determined.
  • the ways to determine the K adjusted prediction modes include but not limited to the following:
  • K second improved prediction modes corresponding to the weight derivation mode of the current block are determined as K adjusted prediction modes.
  • K second improved prediction modes corresponding to each candidate weight derivation mode are determined, so that according to the first After the weight derivation mode of the current block is determined, K second improved prediction modes corresponding to the weight derivation mode of the current block may be directly determined as K adjusted prediction modes.
  • the K second improved prediction modes corresponding to the weight derivation mode of the current block are improved to obtain the K adjusted prediction modes.
  • the above S103 -A includes the following steps:
  • S103-A3 includes but is not limited to the following examples:
  • Example 1 Determine at least one prediction mode related to the weight derivation mode of the current block.
  • the at least one prediction mode may be a prediction mode whose prediction angle is parallel or approximately parallel to the boundary line of the weight derivation mode of the current block or a prediction mode whose prediction angle is parallel to the current block’s weight derivation mode.
  • the weights of the blocks are derived from the mode's vertical or nearly vertical prediction modes.
  • the decoder can obtain K adjusted prediction modes according to the cost corresponding to the new prediction mode combination and the costs corresponding to the K second improved prediction modes.
  • the The K prediction modes included in the new prediction mode combination 1 are determined as K adjusted prediction modes, and the new prediction mode combination 1 may include at least one initial prediction mode, or may not include the initial prediction mode.
  • Example 2 through the following steps from S103-A31 to S103-A34, K second prediction modes are improved to obtain K adjusted prediction modes:
  • the above S103-A32 can be executed after the above S103-A31, or it can be executed before the above S103-A31, or it can be executed together with the above S103-A31.
  • A31 is executed synchronously, which is not limited in this embodiment of the present application.
  • the decoding end uses the weight derivation mode of the current block to improve each of the K second improved prediction modes, and obtains the corresponding The adjusted forecasting modes, and then get K adjusted forecasting modes.
  • the process of determining the adjusted prediction mode corresponding to each second improved prediction mode in the K second improved prediction modes is the same.
  • the embodiment of the present application determines the k-th II.
  • the adjusted prediction mode corresponding to the improved prediction mode is described as an example.
  • the decoder first determines P second candidate prediction modes corresponding to the kth second improved prediction mode.
  • the manner in which the decoding end determines the P second candidate prediction modes corresponding to the kth second improved prediction mode includes the following two situations:
  • P intra-frame prediction modes similar to the k-th second improved prediction mode are determined as P second candidate prediction modes. For example, if the kth second improved prediction mode is an angle prediction mode, then the P angle prediction modes that are similar (or similar) to the angle of the kth second improved prediction mode are determined to be corresponding to the kth second improved prediction mode The P second candidate prediction modes of .
  • the k-th second improved prediction mode is an inter-frame prediction mode
  • search according to the k-th second improved prediction mode to obtain P pieces of second motion information, and determine the P pieces of second motion information based on the P pieces of second motion information.
  • P second candidate prediction modes corresponding to the k second improved prediction modes For example, for the kth second improved prediction mode, its motion vector is (xInit, yInit), set a search range such as from xInit-sR to xInit+sR in the horizontal direction, and from yInit-sR to yInit+sR in the vertical direction A rectangular area, where sR can be 2, 4, 8, etc.
  • Each motion vector in the rectangular area can be combined with other information of the kth second improved prediction mode, such as reference frame index and prediction list flag, etc. to determine a motion information, thereby determining a prediction mode, so that the first P second candidate prediction modes corresponding to the k second improved prediction modes.
  • other information of the kth second improved prediction mode such as reference frame index and prediction list flag, etc.
  • one way to obtain the P pieces of second motion information by searching according to the k-th second improved prediction mode is to search in the preset search mode according to the motion vector corresponding to the k-th second improved prediction mode. Search within the range to obtain P offsets, and each offset includes an offset in the first direction (for example, the x direction) and an offset in the second direction (for example, the y direction). Furthermore, according to the M offsets, P pieces of second motion information are obtained. For example, for each of the P offsets, according to the offset, the kth second improved prediction mode corresponds to The motion vector is shifted to obtain a second motion information.
  • the second candidate prediction mode for each of the P second candidate prediction modes, use the second candidate prediction mode to predict the template of the current block to obtain a prediction value, and then, according to the first template weight and The prediction value determines a fifth cost of the first candidate prediction mode with respect to the template.
  • use the second candidate prediction mode to predict the pixel to obtain a predicted value of the pixel, determine the template weight corresponding to the pixel in the first template weight, and set the The product of the predicted value of the pixel and the template weight determines the predicted value of the second candidate prediction mode at the pixel.
  • the predicted value of the second candidate prediction mode at each pixel point in the template can be obtained, and these predicted values form the template predicted value corresponding to the second candidate prediction mode. According to the template predicted value and the reconstructed value of the template, A fifth cost corresponding to the second candidate prediction mode is determined.
  • the fifth cost corresponding to each of the P second candidate prediction modes can be determined, and then, the following S103-A14 is performed.
  • the second candidate prediction mode with the fifth smallest cost among the P second candidate prediction modes is determined as the adjusted prediction mode corresponding to the kth second improved prediction mode.
  • the above embodiment is described by taking the determination of the adjusted prediction mode corresponding to the k-th second improved prediction mode among the K second improved prediction modes as an example. Referring to the above description, each of the K second improved prediction modes can be determined. The adjusted prediction mode corresponding to the second improved prediction mode, and then K adjusted prediction modes are obtained.
  • the decoder first uses P sub-templates to improve K initial prediction modes to obtain K second improved prediction modes corresponding to the weight derivation mode of the current block, and then uses the weight derivation mode of the current block to improve the current block
  • the K second improved prediction modes corresponding to the weight derivation mode of the weight are improved to obtain accurate K adjusted prediction modes, so that when the current block is predicted using the K adjusted prediction modes and the weight derivation mode of the current block, it can be Further improve the accuracy of forecasting.
  • the weight derivation mode of the current block and K adjusted prediction modes can be determined, and then the current block can be predicted by using the weight derivation mode of the current block and the K adjusted prediction modes to obtain the current The predicted value of the block.
  • the ways of determining the prediction value of the current block include but are not limited to the following:
  • the above S103-B includes the following steps:
  • S103-B13 Determine the prediction value of the current block according to the K prediction values and the weight of the prediction value.
  • the weight derivation mode of the current block is used to determine the weight of the prediction value used by the current block.
  • the weight derivation mode of the current block may be a mode of deriving the weight of the prediction value.
  • each weight derivation mode can derive a weight matrix of predictive values; for blocks of the same size, the weight matrices of predictive values derived from different weight derivation modes can be different.
  • the decoder when determining the prediction value of the current block based on the K adjusted prediction modes and the weights of the prediction values, the decoder may first determine K predictions based on the K adjusted prediction modes value; then use the weight of the predicted value to perform weighted average calculation on the K predicted values, and finally the predicted value of the current block can be obtained.
  • determining the weight of the first template and the weight of the predicted value according to the weight derivation mode of the current block are two independent processes without interfering with each other.
  • the present application may also use the second method below to determine the predicted value of the current block.
  • S103-B includes the following steps:
  • S103-B23 Determine the predicted value of the current block according to the K predicted values and the weights of the K predicted values.
  • the weight of the pixels in the merged area is derived according to the weight derivation mode of the current block, and the merged area includes the current block and the template area of the current block, and then the weight corresponding to the current block in the merged area is The weight is determined as the weight of the predicted value, and the weight corresponding to the template region in the merged region is determined as the first template weight. That is to say, in the second method, the template area and the current block are taken as a whole area, and the first template weight and the weight of the predicted value can be derived in one step, which reduces the steps of weight derivation and improves the prediction effect.
  • the above prediction process is performed in units of pixels, and the corresponding weights are also weights corresponding to pixels.
  • each of the K adjusted prediction modes is used to predict a certain pixel A in the current block, and the K adjusted prediction modes with respect to the pixel A are obtained.
  • the weights of the K predictive values of pixel A are determined according to the weight derivation mode of the current block, and the K predictive values are weighted to obtain the predictive value of pixel A.
  • the first predicted value and the second predicted value are weighted according to the weight of the predicted value corresponding to pixel A to obtain the predicted value of pixel A.
  • both the first prediction mode and the second prediction mode are intra-frame prediction modes
  • the first intra-frame prediction mode is used for prediction to obtain the first predicted value
  • the second The intra prediction mode performs prediction to obtain the second prediction value, and weights the first prediction value and the second prediction value according to the weight of the prediction value to obtain the prediction value of the current block.
  • the first intra-frame prediction mode is used to predict the pixel point A to obtain the first predicted value of the pixel point A
  • the second intra-frame prediction mode is used to predict the pixel point A to obtain the second predicted value of the pixel point A
  • the first predicted value and the second predicted value are weighted to obtain the predicted value of the pixel point A.
  • the above S103-B determines the current block according to the K adjusted prediction modes and the weight derivation mode of the current block.
  • the predicted value of a block includes the following steps:
  • the intra prediction mode is used for prediction
  • the first prediction value is obtained
  • the inter prediction mode is used for prediction , to obtain the second prediction value, and weight the first prediction value and the second prediction value according to the weight of the prediction value to obtain the prediction value of the current block.
  • the intra prediction mode is used to predict each point in the current block to obtain the predicted value of each point in the current block, and the predicted value of each point in the current block constitutes the first predicted value of the current block.
  • the inter-frame prediction mode determine a piece of motion information, determine the best matching block of the current block according to the motion information, and determine the best matching block as the second prediction value of the current block. For the weight of the prediction value of each pixel in the current block, a point-by-point weighting operation is performed on the first prediction value and the second prediction value of the current block to obtain the prediction value of the current block. For example, for pixel A in the current block, according to the weight of the prediction value of pixel A, the first prediction value corresponding to pixel A in the first prediction value of the current block is compared with the pixel in the second prediction value of the current block The second predicted value corresponding to point A is weighted to obtain the predicted value of pixel point A.
  • the weights of the predicted values corresponding to any two adjusted prediction modes among the K adjusted prediction modes can be determined according to the weight derivation mode of the current block, and the weights of the predicted values corresponding to any two adjusted prediction modes among the K adjusted prediction modes can be determined.
  • the weights of the predicted values corresponding to the K adjusted prediction modes can be determined according to the preset weight ratio, Assuming that the weight of the predicted value corresponding to the third prediction mode accounts for 1/4 of the total weight of the predicted value in the entire medium, it can be determined that the total weight of the predicted value of the third prediction mode is 2, and the remaining 3 of the total weight of the predicted value /4 is assigned to the first prediction mode and the second prediction mode.
  • the weight of the prediction value corresponding to the first prediction mode is determined to be (3/4)*3, and the weight of the prediction value corresponding to the second prediction mode
  • the weight of the corresponding predicted value is (3/4)*5 for the predicted value of the first prediction mode.
  • the decoding end determines the predicted value of the current block according to the above method.
  • the decoding end obtains the quantization coefficient of the current block by decoding the code stream, performs inverse quantization on the quantization coefficient to obtain the transformation coefficient, and performs inverse transformation on the transformation coefficient to obtain the current block.
  • the residual value of the block The prediction value of the current block is added to the residual value to obtain the reconstruction value of the current block.
  • the decoding end decodes the code stream to determine K initial prediction modes; determine the weight derivation mode of the current block according to the K initial prediction modes; determine the current block according to the weight derivation mode of the current block predicted value of . That is, the present application determines the weight derivation mode of the current block through K initial prediction modes, thereby enriching the determination method of the weight derivation mode.
  • the encoding end does not need to transmit the index of the weight derivation mode in the code stream, and the decoding end can also determine the weight derivation mode of the current block according to the K initial prediction modes, thereby saving codewords and reducing the encoding cost.
  • the prediction method of the present application is introduced above by taking the decoding end as an example, and the following description is made by taking the encoding end as an example.
  • FIG. 19 is a schematic flowchart of a prediction method provided by an embodiment of the present application, and the embodiment of the present application is applied to the video encoder shown in FIG. 1 and FIG. 2 .
  • the method of the embodiment of the present application includes:
  • each coding block may include a first image component, a second image component, and a third image component.
  • the coding block to be predicted can be called a brightness block; or, assuming that the first prediction is performed Two prediction, and the second image component is a chrominance component, that is, the image component to be predicted is a chrominance component, then the coding block to be predicted can be called a chrominance block.
  • the encoding end needs to be in the code stream It is necessary to indicate the weight derivation mode, such as writing the index of the weight derivation mode in the code stream.
  • the weight derivation mode such as writing the index of the weight derivation mode in the code stream.
  • the decoding end in order to reduce the encoding cost, derives the weight derivation mode through the prediction mode. Specifically, the decoder determines K initial prediction modes by decoding the code stream, and then determines the weight derivation mode of the current block according to the K initial prediction modes, and then determines the prediction value of the current block according to the weight derivation mode of the current block .
  • the encoding end also adopts the same prediction method as the decoding end, that is, determines K initial prediction modes, and determines the weight derivation mode of the current block according to the K initial prediction modes, and then determines the current block derivation mode according to the weight derivation mode of the current block. The predicted value of the block.
  • the weight derivation mode of the current block is denoted as the weight derivation mode of the current block.
  • the above K initial prediction modes of the current block include the following examples:
  • Example 1 the above K initial prediction modes are all intra-frame prediction modes.
  • Example 2 the above K initial prediction modes are all inter-frame prediction modes.
  • Example 3 among the aforementioned K initial prediction modes, at least one is an intra-frame prediction mode, and at least one is an inter-frame prediction mode.
  • Example 4 among the above K initial prediction modes, at least one is an intra prediction mode, and at least one is a non-inter and non-intra prediction mode, for example, an intra-block copy IBC prediction mode or a palette prediction mode.
  • Example 5 among the aforementioned K initial prediction modes, at least one is an inter prediction mode, and at least one is a non-inter and non-intra prediction mode, for example, an IBC prediction mode or a palette prediction mode.
  • Example 6 none of the above K initial prediction modes is an intra prediction mode or an inter prediction mode, for example, one is an IBC prediction mode, one is a palette prediction mode, and so on.
  • the methods for determining the K initial prediction modes in S201 above include but are not limited to the following:
  • Mode 1 the above K initial prediction modes are preset K prediction modes.
  • S201 includes the steps from S201-A1 to S201-A2:
  • S201-A Determine a candidate prediction mode list, where the candidate prediction mode list includes at least two candidate prediction modes;
  • the candidate prediction modes included in the list of candidate prediction modes are all intra-frame prediction modes, for example including direct current (Direct Current, DC) at least one of a planar mode, a planar (PLANAR) mode, an angle mode, and the like.
  • the above candidate prediction mode list may also include intra prediction modes in the MPM list.
  • the candidate prediction modes included in the candidate prediction mode list are all inter-frame prediction modes, for example, include skip, merge, normal inter-frame At least one of prediction modes and the like.
  • the candidate prediction mode list includes at least one intra prediction mode and at least one inter prediction mode.
  • the above list of candidate prediction modes may also include modes such as IBC and palette.
  • the above list of candidate prediction modes may also include unidirectional prediction, bidirectional prediction, multi-hypothesis prediction, and the like.
  • the present application does not limit the types and numbers of the candidate prediction modes included in the above list of candidate prediction modes.
  • the embodiment of the present application does not limit the manner of determining the candidate prediction mode list.
  • the candidate prediction modes included in the above list of candidate prediction modes are preset modes.
  • the above list of candidate prediction modes is an MPM list.
  • the above list of candidate prediction modes is a set of prediction modes determined according to some rules, such as equidistant screening.
  • both the encoding end and the decoding end default that the candidate prediction modes in the above candidate prediction mode list are determined by template matching.
  • the encoding end uses the prediction mode to predict the template of the current block to obtain the template
  • the cost of the prediction mode such as SAD cost, SATD cost or MSE cost.
  • the encoder can sort the prediction modes according to the cost of each prediction mode to obtain a list of candidate prediction modes, for example, determine multiple prediction modes with the lowest cost as candidate prediction modes to form a list of candidate prediction modes.
  • K candidate prediction modes are selected from the candidate prediction mode list as K initial prediction modes.
  • the encoding end randomly selects K candidate prediction modes from the candidate prediction mode list as K initial prediction modes.
  • the encoder determines the K candidate prediction modes with the lowest cost in the candidate prediction mode list as the K initial prediction modes.
  • the encoder After the encoder determines the K initial prediction modes, it writes the prediction mode index in the code stream.
  • the prediction mode index can be the index of at least one of the K initial prediction modes.
  • the prediction mode index is the K initial prediction mode index.
  • the decoding end decodes the code stream to obtain the prediction mode index, and determines K initial prediction modes from the determined candidate prediction mode list according to the prediction mode index.
  • the encoder determines K initial prediction modes according to the above steps, determines the weight derivation mode of the current block according to the K initial prediction modes, and then predicts the current block according to the weight derivation mode of the current block, Determine the predicted value for the current block.
  • the encoder before determining the K initial prediction modes, the encoder first needs to determine whether the current block uses K different prediction modes for weighted prediction processing. If the encoder determines that the current block uses K different prediction modes for weighted prediction processing, it executes the above S201 to determine K initial prediction modes.
  • the encoding end may determine whether the current block uses two different prediction modes for weighted prediction processing by determining the prediction mode parameter of the current block.
  • the prediction mode parameter may indicate whether the current block can use GPM mode or AWP mode, that is, indicate whether the current block can use K different prediction modes for prediction processing.
  • the prediction mode parameter can be understood as a flag indicating whether the GPM mode or the AWP mode is used.
  • the encoding end may use a variable as the prediction mode parameter, so that the setting of the prediction mode parameter may be realized by setting the value of the variable.
  • the encoder can set the value of the prediction mode parameter to indicate that the current block uses GPM mode or AWP mode.
  • the encoder can set the variable The value of is set to 1.
  • the encoding end can set the value of the prediction mode parameter to indicate that the current block does not use GPM mode or AWP mode, specifically, the encoding end can Set the variable value to 0. Furthermore, in the embodiment of the present application, after the encoding end completes the setting of the prediction mode parameters, it can write the prediction mode parameters into the code stream and transmit it to the decoding end, so that the decoding end can analyze the code stream Get the prediction mode parameters.
  • the encoder determines whether the current block uses GPM mode or AWP mode according to the prediction mode parameter. If the current block uses GPM mode or AWP mode, that is, when K different prediction modes are used for prediction processing, K initial prediction modes are determined. .
  • the GPM mode or the AWP mode is a prediction method, specifically, K different prediction modes are determined for the current block, and then determined according to the K different prediction modes. K predicted values, and then the weights can be determined again, and the K predicted values are combined according to the weights, and finally the predicted value of the current block can be obtained.
  • the size of the current block can be restricted. It can be understood that since the prediction method proposed in the embodiment of the present application needs to use K different prediction modes to generate K prediction values, and then weight them according to the weight to obtain the prediction value of the current block, in order to reduce the complexity, the compression For the trade-off between performance and complexity, in the embodiment of the present application, the GPM mode or AWP mode may not be used for blocks of certain sizes. Therefore, in this application, the encoder can first determine the size parameter of the current block, and then determine whether the current block uses the GPM mode or the AWP mode according to the size parameter.
  • the size parameter of the current block may include the height and width of the current block, therefore, the encoder may use the height and width of the current block to restrict the use of GPM mode or AWP mode.
  • the encoder may use the height and width of the current block to restrict the use of GPM mode or AWP mode.
  • the width is greater than the first threshold and the height is greater than the second threshold, it is determined that the current block uses the GPM mode or the AWP mode. It can be seen that a possible restriction is to use GPM mode or AWP mode only when the width of the block is greater than (or greater than or equal to) the first threshold and the height of the block is greater than (or greater than or equal to) the second threshold.
  • the values of the first threshold and the second threshold may be 8, 16, 32, etc., and the first threshold may be equal to the second threshold.
  • the width is smaller than the third threshold and the height is larger than the fourth threshold, it is determined that the current block uses the GPM mode or the AWP mode. It can be seen that a possible restriction is to use the AWP mode only when the width of the block is smaller than (or smaller than or equal to) the third threshold and the height of the block is larger than (or larger than or equal to) the fourth threshold.
  • the values of the third threshold and the fourth threshold may be 8, 16, 32, etc., and the third threshold may be equal to the fourth threshold.
  • the limitation of the size of the block that can use the GPM mode or the AWP mode can also be realized through the limitation of the pixel parameters.
  • the encoding end may first determine the pixel parameters of the current block, and then further judge whether the current block can use the GPM mode or the AWP mode according to the pixel parameters and the fifth threshold. It can be seen that a possible restriction is to use the GPM mode or the AWP mode only when the pixel number of the block is greater than (or greater than or equal to) the fifth threshold. Wherein, the value of the fifth threshold may be 8, 16, 32 and so on. That is to say, in this application, the current block can use the GPM mode or the AWP mode only if the size parameter of the current block meets the size requirement.
  • this application there may be a frame-level flag to determine whether the current frame to be decoded uses the technical solution of this application. For example, it can be configured that an intra frame (such as an I frame) uses this application, and an inter frame (such as a B frame, P frame) does not use this application. Or it can be configured that intra frames do not use this application, and inter frames use this application. Or you can configure some inter frames to use this application and some inter frames not to use this application. Inter frames can also use intra prediction, thus inter frames are also likely to use this application.
  • an intra frame such as an I frame
  • an inter frame such as a B frame, P frame
  • a flag below the frame level and above the CU level such as tile, slice, patch, LCU, etc.
  • the weight derivation mode is used to determine the weight used by the current block.
  • the weight derivation mode may be a mode for deriving weights.
  • each weight export mode can export a weight matrix; for blocks of the same size, different weight export modes export different weight matrices.
  • AWP has 56 weight derivation modes
  • GPM has 64 weight derivation modes.
  • the finally determined weight derivation mode for prediction of the current block is recorded as the weight derivation mode of the current block.
  • the encoding end determines the weight derivation mode of the current block according to the K initial prediction modes determined above. In some cases, the encoding end may not write information related to the weight derivation mode of the current block in the code stream , thus saving codewords and reducing the encoding cost.
  • the embodiment of the present application does not limit the manner in which the encoding end determines the weight derivation mode of the current block according to the K initial prediction modes.
  • the encoder determines the weight derivation mode of the current block based on the template of the current block and according to the K initial prediction modes.
  • GPM has 64 weight derivation modes, and the encoder estimates which weight derivation modes are more likely to be selected and which weight derivation modes are less likely to be selected by analyzing the template.
  • the encoder estimates which weight derivation modes are more likely to be selected and which weight derivation modes are less likely to be selected by analyzing the template.
  • the edge on the template can be used to infer the edge of the current block.
  • An example is as follows, given 2 motion information, according to the 2 motion information, the predicted values on the 2 templates are respectively obtained, which are recorded as the first template predicted value and the second template predicted value, and the template weight is derived according to a certain weight derivation mode , according to the first template prediction value, the second template prediction value and the template weight, determine the template prediction value corresponding to the weight derivation mode, because the reconstruction value of the template is available, so the template prediction value corresponding to the weight derivation mode and The reconstruction value of the template can get the cost of the weight derived mode on the template. This makes it possible to estimate the probability of each weight-derived mode being selected based on its cost on the template. One is to sort the weight-derived modes according to their cost on the template.
  • the mode with the least cost comes first, and the mode with the most cost follows. Because it is considered that the sorted weight derivation patterns are roughly arranged from high probability to low probability, so the method of variable length coding can be used for processing.
  • One possible implementation is to build a list of weight derivation modes, the encoder writes the weight derivation mode of the current block selected by the current block into the code stream according to the index in the list, and the decoder parses the index in the code stream, from The list determines the weight export mode for the current block.
  • Template matching uses the correlation between adjacent pixels to use some areas around the current block as templates. When encoding and decoding the current block, its left side and upper side have been decoded according to the encoding sequence. During inter-frame prediction, the best matching position of the template is found in the reference frame to determine the motion information or motion vector of the current block. During intra-frame prediction, a template is used to determine the intra-frame prediction mode of the current block.
  • the present application does not limit the specific shape of the template of the current block.
  • the template includes at least one of an upper decoded region, a left decoded region, and an upper left decoded region of the current block.
  • the width of the upper decoded area is the same as the width of the current block
  • the height of the left decoded area is the same as the height of the current block
  • the width of the upper left decoded area is the same as the width of the left decoded area
  • the upper left The height of the decoded region is the same as the height of the decoded region above.
  • the K prediction modes include the first prediction mode and the second prediction mode
  • the templates corresponding to the first prediction mode and the second prediction mode are the upper decoded blocks of the current block. area, or the decoded area on the left side of the current block, or the decoded area on the left side and the upper side of the current block.
  • the white area of the weight matrix of the current block is the weight corresponding to the predicted value of the first prediction mode
  • the black area is the weight corresponding to the predicted value of the second prediction mode. the weight of.
  • the first template corresponding to the first prediction mode is the upper decoded area of the current block
  • the second template corresponding to the second prediction mode is the left decoded area of the current block, however, the same as the second prediction mode
  • the adjacent template includes not only the left area, but also part of the upper decoded area. Therefore, the division of templates in the prior art is not fine enough, and the weight derivation mode of the current block cannot be accurately derived based on the template.
  • this application can realize a finer division of the template through the weight derivation mode.
  • the template on one side of the dividing line is recorded as the first template
  • the template on the other side of the dividing line is recorded as the second template.
  • the first template corresponds to the first prediction mode
  • the second template corresponds to the second prediction mode
  • the weight of the first template is determined according to the weight corresponding to the first prediction mode
  • the weight of the second template is determined according to the weight corresponding to the second prediction mode
  • the weight of the first template is the same as the weight corresponding to the first prediction mode
  • the weight of the second template is the same as the weight corresponding to the second prediction mode.
  • the first template and the second template divided according to the above method may not be rectangular.
  • the first template and the second template have hypotenuses, and the cost calculation for irregular templates is more complicated.
  • the weight matrix can be directly extended to the template area, for example, extended to the left and upward, and the weight matrix can be overlaid on the template to determine the weight of the template.
  • the weight matrix can be directly extended to the template area, for example, extended to the left and upward, and the weight matrix can be overlaid on the template to determine the weight of the template.
  • the area in the inverted L-shaped frame on the left, upper left, and upper side is the template area, and the rectangular area at the lower right is the current block, so expand it to the upper left
  • the weight matrix of becomes the weight matrix of the template.
  • the weight derivation mode is used to simulate the template division, or the weight derivation mode is used to determine the template weight, and the template matching cost is determined according to the template weight. Since the division of templates may be different under different block shapes, this application does not need to set various rules for blocks of various shapes, for example, it is not necessary to set the correspondence between prediction modes and templates shown in Table 5. In terms of the accuracy of "division", the simulated division of the present application is also more precise and accurate than the method of only distinguishing the left area and the upper area.
  • the left area and the upper area of the current block shown in FIG. 17B may be used as the template of the current block, without considering the upper left matrix area.
  • Y can be used in YUV format because Y is dominant, and of course all components can be used.
  • Y can be used in YUV format because Y is dominant, and of course all components can be used.
  • template matching is performed on the Y component as an example, and reference can be made to other components.
  • deriving the template weight according to the weight derivation mode may be combined with the derivation process of the weight of the predicted value, for example, simultaneously deriving the weight of the template and the weight of the predicted value.
  • the weight of the predicted value can be understood as the weight corresponding to the predicted value, for example, use the first prediction mode to obtain the first predicted value, use the second prediction mode to obtain the second predicted value, and determine the first weight of the first predicted value according to the weight derivation mode , determine the second weight of the second predictor according to the weight derivation mode, and determine the sum of the product of the first predictor and the first weight and the product of the second predictor and the second weight as the predictor of the current block.
  • the first weight and the second weight above are referred to as the weight of the predicted value.
  • determining the template weight according to the weight derivation mode includes the following steps:
  • Step A determine the angle index and the distance index according to the weight derivation mode
  • Step B Determine the template weight according to the angle index, the distance index and the size of the template.
  • the present application may derive the template weight in the same manner as the weight of the predicted value.
  • an angle index and a distance index are determined according to the weight derivation mode, where the angle index can be understood as the boundary angle index of each weight derived by the weight derivation mode.
  • the angle index and distance index corresponding to the weight derivation mode can be determined according to the above Table 2. For example, if the weight derivation mode is 27, the corresponding angle index is 12 and the distance index is 3. Then, according to the angle index, the distance index and the size of the template, the template weight is determined.
  • the methods of exporting the template weight according to the weight export mode in the above step B include but are not limited to the following methods:
  • the first way is to directly determine the template weight according to the angle index, the distance index and the size of the template.
  • the above step B includes the following steps from step B11 to step B13:
  • Step B11 according to the angle index, the distance index and the size of the template, determine the first parameter of the pixel in the template;
  • Step B12 according to the first parameter of the pixel in the template, determine the weight of the pixel in the template
  • Step B13 Determine the template weight according to the weight of the pixel points in the template.
  • the weight of the pixel points in the template is determined according to the angle index, the distance index, the size of the template and the size of the current block, and then the weight matrix composed of the weights of each point in the template is determined as the template weight.
  • the first parameter of this application is used to determine the weights.
  • the first parameter is also referred to as a weight index.
  • the offset and the first parameter may be determined according to the manner of the above-mentioned embodiment.
  • the offset and the first parameter may be determined according to the manner of the above-mentioned embodiment.
  • the weight at the midpoint (x, y) of the template is determined according to the weightIdx.
  • the ways of determining the weight of the pixel in the template include but not limited to the following:
  • Mode 1 according to the first parameter of the pixel in the template, determine the second parameter of the pixel in the template; according to the second parameter of the pixel in the template, determine the weight of the pixel in the template.
  • the second parameter is also used to determine the weight.
  • the above-mentioned second parameter is also referred to as a weight index under the first component, and the first component may be a luma component, a chrominance component, and the like.
  • the weight of the pixel in the template is determined.
  • the weight of the pixel point in the template is limited to the first preset value or the second preset value, that is to say, the weight of the pixel point in the template is either the first preset value value, or the second preset value, thereby reducing the computational complexity of the template weight.
  • the present application does not limit specific values of the first preset value and the second preset value.
  • the first preset value is 1.
  • the second preset value is 0.
  • the weight of each point in the template is determined through the weight export mode, and the weight matrix composed of the weight of each point in the template is used as the template weight.
  • Method 2 Determine the weights of the current block and the template according to the weight derivation mode. That is to say, in this method 2, take the merged area composed of the current block and the template as a whole, and derive the weight of the pixels in the merged area according to the weight export mode. Weights. Based on this, the above step B includes the following steps B21 and S202-B22 steps:
  • Step B21 according to the angle index, the distance index, the size of the template and the size of the current block, determine the weight of the pixels in the merged area formed by the current block and the template;
  • Step B22 Determine the template weight according to the size of the template and the weight of the pixels in the merged area.
  • the current block and the template are taken as a whole, according to the angle index, the distance index, the size of the template and the size of the current block, the weight of the pixels in the merged area composed of the current block and the template is determined, and then according to the weight of the template Size, determine the weight corresponding to the template among the weights of the pixels in the merged area as the template weight, for example, as shown in Figure 18, determine the weight corresponding to the L-shaped template area in the merged area as the template weight.
  • the weight of the predicted value performs the subsequent forecasting process without re-determining the weight of the predicted value, thereby reducing the number of forecasting steps and improving the forecasting efficiency.
  • the present application does not limit the specific implementation manner of determining the weight of the pixels in the merged area formed by the current block and the template according to the angle index, the distance index, the size of the template, and the size of the current block.
  • the determination of the weights of the pixels in the merged area in the above step B21 includes the following steps from step B211 to step B212:
  • Step B211 according to the angle index, the distance index and the size of the merged area, determine the first parameter of the pixel in the merged area;
  • Step B212 according to the first parameter of the pixels in the merged area, determine the weight of the pixels in the merged area.
  • the weight of the pixel points in the merged area is determined, and the weight of each point in the merged area forms a weight matrix.
  • the ways of determining the weight of the pixels in the merged region include but not limited to the following:
  • Method 1 according to the first parameter of the pixels in the merged area, determine the second parameter of the pixels in the merged area; according to the second parameter of the pixels in the merged area, determine the weight of the pixels in the merged area.
  • the weight of the pixels in the merged area is determined.
  • the weight of the pixels in the merged area is the first preset value or the second preset value.
  • the weight of the pixel in the merged area is limited to the first preset value or the second preset value, that is to say, the weight of the pixel in the merged area is either the first preset value or the second preset value.
  • a preset value, or a second preset value so as to reduce the computational complexity of merging region weights.
  • the present application does not limit specific values of the first preset value and the second preset value.
  • the first preset value is 1.
  • the second preset value is 0.
  • the weight of each point in the merged area composed of the current block and the template is determined through the weight derivation mode, and then according to the size of the template, the weight corresponding to the template in the merged area is determined as the template weight, and the method 2
  • the weight of the predicted value is determined at the same time, and there is no need to perform subsequent steps of determining the weight of the predicted value, which reduces the number of prediction steps and improves the prediction efficiency.
  • the encoding end derives the mode according to the weight of the current block, and the ways of determining the predicted value of the current block include but not limited to the following:
  • the encoding end determines the predicted value of the current block according to the weight derivation mode of the current block and K initial prediction modes. Specifically, the encoding end determines the weight of the prediction value according to the weight derivation mode of the current block, and then according to the K initial prediction modes mode to predict the current block to obtain K predicted values, and then use the weight of the predicted values to perform weighted average calculation on the K predicted values to obtain the predicted value of the current block.
  • the weight of the prediction value derived according to the weight derivation mode of the current block can be understood as deriving the weight corresponding to each pixel in the current block, or can be understood as deriving the weight matrix corresponding to the current block.
  • K predicted values corresponding to each pixel in the current block may be determined, and according to the K predicted values corresponding to each pixel and the weight of the predicted value, A prediction value corresponding to each pixel is determined, and the prediction value corresponding to each pixel in the current block constitutes a prediction value of the current block.
  • determining the predicted value of the current block based on the weight of the predicted value it can also be performed according to the block, for example, determine K predicted values of the current block, and calculate the K predicted values of the current block according to the weight matrix of the current block Weighted to get the predicted value of the current block.
  • the encoder can use the weight derivation mode of the current block to improve the above K initial prediction modes.
  • the above S203 includes the following steps of S203-A and S203-B:
  • S203-B Determine the prediction value of the current block according to the K adjusted prediction modes and the weight derivation mode of the current block.
  • the GPM has K+1 elements, that is, one weight derivation mode and K prediction modes. Since one weight derivation mode and K prediction modes jointly generate a prediction block, this prediction block acts on the current block. They are related. To give a few examples, say the current block contains the edges of 2 relatively moving objects, which is an ideal scenario for inter-frame GPM. In theory, this "division" should occur at the edge of the object, but in fact there are limited possibilities for "division", and it is impossible to cover any edge. Sometimes a similar "division" is selected, so there may be more than one similar "division". One, which one to choose depends on which "division" and the combination of the two prediction modes have the best results.
  • the weight derivation mode can be used to determine the prediction mode, and the prediction mode can also be used to determine the weight derivation mode.
  • some schema information can also be refined.
  • the so-called improvement is to obtain a better model information by collecting more relevant information and using more calculations on the basis of an initial model information.
  • refine is an operation on the encoding side.
  • the motion information can also be improved and refined based on the template. That is, on the basis of the initial motion information, a small-scale search is performed at the encoding end, and the motion information is optimized according to the degree of matching between the searched motion information and the template.
  • One understanding of the so-called matching degree between motion information and the template is the matching degree between the predicted value of the motion information on the template and the reconstructed value of the template.
  • the weight derivation mode can be determined according to the prediction mode, and then return to refine the prediction mode according to the weight derivation mode. Or first determine the prediction mode according to the weight derivation mode, and then derive the mode according to the prediction mode refine weight.
  • the two can also be implemented alternately, if the complexity allows.
  • Templates can be used as a link to implement these methods, and further, template weights can be used as a link to implement these methods. That is, the cost on the template is calculated according to the predicted value of the prediction mode on the template and the sub-template or template weight corresponding to the weight derivation mode. The cost on the template is used to derive or determine the prediction mode or weight derivation mode.
  • the weight derivation mode of the current block is determined according to the K initial prediction modes, and then the K initial prediction modes are improved by using the weight derivation mode of the current block to obtain K adjusted prediction modes , and then determine the prediction value of the current block according to the weight derivation mode of the current block and the K adjusted prediction modes.
  • the specific process of determining the prediction value of the current block according to the K adjusted prediction modes and the weight derivation mode of the current block can refer to the following to predict the current block according to the weight derivation mode and K prediction modes, and determine the prediction of the current block The specific description of the value will not be repeated here.
  • the above S202 includes the following steps from S202-A to S202-C:
  • i is a positive integer from 1 to N;
  • the encoder first determines N candidate weight derivation modes, the N candidate weight derivation modes form a list of weight derivation modes, and the encoder calculates the corresponding The weight derivation mode of the current block is determined from the N candidate weight derivation modes according to the first cost.
  • the candidate weight derivation mode with the smallest cost among the N candidate weight derivation modes is determined as the weight derivation mode of the current block.
  • the encoder writes the weight derivation mode index corresponding to the weight derivation mode of the current block into the code stream, so that the decoder determines the current block from the determined candidate weight derivation mode list according to the weight derivation mode index.
  • the block's weight export mode is the weight derivation mode index corresponding to the weight derivation mode of the current block into the code stream, so that the decoder determines the current block from the determined candidate weight derivation mode list according to the weight derivation mode index.
  • the foregoing first cost may be a cost such as SAD, SATD, or SEE.
  • the process for the encoder to determine the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes is the same. Take the price as an example.
  • the ways of determining the first cost corresponding to the i-th candidate weight derivation mode include but are not limited to the following:
  • Method 1 Use the i-th candidate weight derivation mode and K initial prediction modes to predict the template of the current block to obtain the first predicted value of the template; determine the i-th The first cost corresponding to the weight export mode.
  • the template weight is determined according to the i-th candidate weight derivation mode, and K initial prediction modes are used to predict the template to obtain K predicted values of the template, and the template weight is used to weight the K predicted values corresponding to the template to obtain
  • the predicted value of the template for the convenience of description, the predicted value of the template is recorded as the first predicted value of the template.
  • the prediction cost of the i-th candidate weight derivation mode with respect to the template is determined, and the prediction cost is recorded as the first cost corresponding to the i-th weight derivation mode.
  • the first cost corresponding to each candidate weight derivation mode among the N candidate weight derivation modes can be determined, and then the candidate weight derivation mode with the smallest first cost among the N candidate weight derivation modes is determined as the weight of the current block Derivation mode, or sort the N candidate weight derivation modes according to the first costs corresponding to the N candidate weight derivation modes to obtain a list of candidate weight derivation modes, and index the weight derivation mode in the candidate weight derivation mode list corresponding to the candidate weight Export mode, determined as the weight export mode of the current block.
  • determining K adjusted prediction modes includes the following steps:
  • S203-A1 includes but is not limited to the following examples:
  • Example 1 Determine at least one prediction mode related to the weight derivation mode of the current block.
  • the at least one prediction mode may be a prediction mode whose prediction angle is parallel or approximately parallel to the boundary line of the weight derivation mode of the current block or a prediction mode whose prediction angle is parallel to the current block’s weight derivation mode.
  • the weights of the blocks are derived from the mode's vertical or nearly vertical prediction modes.
  • the combination includes K prediction modes.
  • the K prediction modes included in the new prediction mode combination and the weight derivation mode of the current block For each new prediction mode combination, use the K prediction modes included in the new prediction mode combination and the weight derivation mode of the current block to predict the target, and obtain the predicted value of the template. According to The predicted value of the template and the reconstructed value of the template determine the cost corresponding to the new prediction mode combination, and then the cost corresponding to each new prediction mode combination in the plurality of new prediction mode combinations formed above can be obtained. Similarly, the weight derivation mode of the current block and the K initial prediction modes are used to predict the current block, and the costs corresponding to the K initial prediction modes are determined. In this way, the encoder can obtain K adjusted prediction modes according to the cost corresponding to the new prediction mode combination and the costs corresponding to the K initial prediction modes.
  • the new The K prediction modes included in the prediction mode combination 1 are determined as K adjusted prediction modes, and the new prediction mode combination 1 may include at least one initial prediction mode, or may not include the initial prediction mode.
  • Example 2 through the following steps from S203-A11 to S203-A14, K initial prediction modes are improved to obtain K adjusted prediction modes:
  • the specific process of determining the weight of the first template can refer to the specific description of the derivation embodiment of the template weight below, which will not be repeated here.
  • M is a positive integer greater than 1
  • j is from 1 to K positive integer of .
  • the above S203-A12 can be executed after the above S203-A11, or it can be executed before the above S203-A11, or it can be executed together with the above S203-A11.
  • A11 is executed synchronously, which is not limited in this embodiment of the present application.
  • the encoder uses the weight derivation mode of the current block to improve each of the K initial prediction modes, and obtains the adjusted prediction mode corresponding to each of the K initial prediction modes, and then obtains K adjusted prediction modes.
  • the process of determining the adjusted prediction mode corresponding to each of the K initial prediction modes is the same.
  • the adjusted Forecast mode is used as an example.
  • the encoder first determines M first candidate prediction modes corresponding to the jth initial prediction mode.
  • the manner in which the encoding end determines the M first candidate prediction modes corresponding to the jth initial prediction mode includes the following two situations:
  • M intra-frame prediction modes similar to the j-th initial prediction mode are determined as M first candidate prediction modes. For example, if the jth initial prediction mode is an angle prediction mode, M angle prediction modes that are similar (or similar) to the angle of the jth initial prediction mode are used to determine the M first candidates corresponding to the jth initial prediction mode predictive mode.
  • the j-th initial prediction mode is an inter-frame prediction mode
  • search according to the j-th initial prediction mode to obtain M pieces of first motion information
  • Corresponding M first candidate prediction modes For example, for the jth initial prediction mode, its motion vector is (xInit, yInit), set a search range such as a rectangular area from xInit-sR to xInit+sR in the horizontal direction, and from yInit-sR to yInit+sR in the vertical direction , where sR can be 2, 4, 8, etc.
  • Each motion vector in the rectangular area can be combined with other information of the jth initial prediction mode, such as reference frame index and prediction list flag, etc. to determine a motion information, thereby determining a prediction mode, so that the jth can be obtained M first candidate prediction modes corresponding to the prediction modes.
  • other information of the jth initial prediction mode such as reference frame index and prediction list flag, etc.
  • the above searching according to the jth initial prediction mode to obtain M pieces of first motion information includes: searching according to the ith initial prediction mode to obtain M offsets; according to the M offsets , to obtain M pieces of first motion information.
  • the first candidate prediction mode uses the first candidate prediction mode to predict the template of the current block to obtain a prediction value, and then, according to the first template weight and The prediction value determines a second cost of the first candidate prediction mode with respect to the template.
  • the calculation of the second cost of a first candidate prediction mode with respect to a template among the M first candidate prediction modes is taken as an example for illustration below.
  • the i-th pixel in the template can be understood as any point in the template, that is, the process of determining the second cost of each point in the template is the same, Just refer to the i-th pixel.
  • the first candidate prediction mode is used to predict the template to obtain the predicted value of the template, and the predicted value corresponding to the i-th pixel in the predicted value of the template is recorded as the i-th predicted value, and the i-th
  • the reconstruction value corresponding to the pixel in the reconstruction value of the template is recorded as the i-th reconstruction value, and then according to the i-th predicted value and the i-th reconstruction value, and the weight of the i-th pixel in the first template weight, determine The second cost of the first candidate prediction mode at the i-th pixel.
  • the second cost of the first candidate prediction mode at each point or multiple points in the template is determined, and then the first candidate prediction mode is determined according to the second cost of each point or multiple points in the template.
  • the second cost of the prediction model with respect to the template For example, the sum of the second costs of the first candidate prediction mode at the pixel points in the template is determined as the second cost of the first candidate prediction mode with respect to the template.
  • the second cost of the first candidate prediction mode at the ith pixel point (x, y) in the template can be determined according to the above formula (1).
  • the above takes SAD as an example to determine the second cost of the first candidate prediction mode with respect to the template.
  • the second cost of the first candidate prediction mode with respect to the template can also be determined according to cost calculation methods such as SATD and MSE .
  • the second cost corresponding to each of the M first candidate prediction modes can be determined, and then, the following S203-A14 is performed.
  • the first candidate prediction mode with the smallest second cost among the M first candidate prediction modes is determined as the adjusted prediction mode corresponding to the jth initial prediction mode.
  • the above embodiment is described by taking determining the adjusted prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes as an example. Referring to the above description, the adjustment corresponding to each initial prediction mode among the K initial prediction modes can be determined. prediction mode, and then get K adjusted prediction modes.
  • the encoder first determines the weight derivation mode of the current block according to the K initial prediction modes, and then improves the K initial prediction modes according to the weight derivation mode of the current block, for example, through the above example 1 or example 2
  • the method is improved to obtain K adjusted prediction modes, and then the current block is predicted by using the weight derivation mode of the current block and the K adjusted prediction modes to obtain the prediction value of the current block.
  • the encoder in addition to using the method 1 above to determine the weight derivation mode of the current block and the K adjusted prediction modes, the encoder can also determine the weight derivation mode of the current block and the K adjusted prediction modes through the following method 2 predictive mode.
  • the second way is to integrate the improvement of the prediction mode and the determination of the weight derivation mode of the current block together, and proceed synchronously.
  • determine the i-th The first cost corresponding to the candidate weight derivation mode includes the following steps from S202-B11 to S202-B12:
  • the second method for the i-th candidate weight derivation mode among the N candidate prediction modes, first use the i-th candidate weight derivation mode to improve the K initial prediction modes to obtain the i-th candidate weight derivation mode
  • the corresponding K improved prediction modes for the convenience of description, the K improved prediction modes corresponding to the i-th candidate weight derivation mode are recorded as K first improved prediction modes.
  • the i-th candidate weight derivation mode, and the K first improved prediction modes corresponding to the i-th candidate weight derivation mode determine the first cost corresponding to the i-th candidate weight derivation mode, so that a more accurate current
  • the weight of the block derives the mode and K adjusted prediction modes.
  • S202-B11 includes but is not limited to the following examples:
  • the at least one prediction mode may be a prediction mode whose prediction angle is parallel or approximately parallel to the boundary line of the i-th candidate weight derivation mode or a prediction angle A prediction mode that is vertical or approximately perpendicular to the boundary of the i-th candidate weight derivation mode.
  • each new prediction The mode combination includes K prediction modes, and for each new prediction mode combination, use the K prediction modes included in the new prediction mode combination and the i-th candidate weight derivation mode to predict the template, and obtain the predicted value of the template , according to the predicted value of the template and the reconstruction value of the template, determine the cost corresponding to the new prediction mode combination, and then obtain the cost corresponding to each new prediction mode combination in the plurality of new prediction mode combinations formed above.
  • the encoder can obtain K first improved prediction modes according to the cost corresponding to the new prediction mode combination and the costs corresponding to the K initial prediction modes. For example, if the cost corresponding to the new prediction mode combination 1 is the smallest, then the The K prediction modes included in the new prediction mode combination 1 are determined as K first improved prediction modes, and the new prediction mode combination 1 may include at least one initial prediction mode, or may not include the initial prediction mode.
  • Example 2 through the following steps from S202-B11-1 to S202-B11-4, K initial prediction modes are improved to obtain K first improved prediction modes corresponding to the i-th candidate weight derivation mode:
  • the specific process of determining the second template weight can refer to the specific description of the following template weight derivation embodiment, which will not be repeated here.
  • M is a positive integer
  • j is a number from 1 to K positive integer.
  • the above S202-B11-2 can be executed after the above S202-B11-1, or after S202-B11 Execute before -1, or execute synchronously with the above S202-B11-1, which is not limited in this embodiment of the present application.
  • the encoder uses the i-th candidate weight derivation mode to improve each of the K initial prediction modes, and obtains each of the K initial prediction modes under the i-th candidate weight derivation mode.
  • the first improved prediction mode corresponding to the mode, and then K first improved prediction modes are obtained.
  • the process of determining the first improved prediction mode corresponding to each initial prediction mode in the K initial prediction modes is the same.
  • An example of improving the prediction model will be described.
  • the encoder first determines M first candidate prediction modes corresponding to the jth initial prediction mode.
  • the manner in which the encoding end determines the M first candidate prediction modes corresponding to the jth initial prediction mode includes the following two situations:
  • the above searching according to the jth initial prediction mode to obtain M pieces of first motion information includes: searching according to the ith initial prediction mode to obtain M offsets; according to the M offsets , to obtain M pieces of first motion information.
  • the first candidate prediction mode uses the first candidate prediction mode to predict the template of the current block to obtain a prediction value, and then, according to the second template weight and The prediction value determines a third cost of the first candidate prediction mode with respect to the template.
  • the following takes the calculation of the third cost of a first candidate prediction mode with respect to a template among the M first candidate prediction modes as an example for illustration.
  • the i-th pixel in the template can be understood as any point in the template, that is, the process of determining the third cost of each point in the template is the same, Just refer to the i-th pixel.
  • the first candidate prediction mode is used to predict the template to obtain the predicted value of the template, and the predicted value corresponding to the i-th pixel in the predicted value of the template is recorded as the i-th predicted value, and the i-th
  • the reconstruction value corresponding to the pixel point in the reconstruction value of the template is recorded as the i-th reconstruction value, and then according to the i-th predicted value and the i-th reconstruction value, and the weight of the i-th pixel point in the second template weight, determine The third cost of the first candidate prediction mode at the i-th pixel.
  • the third cost of the first candidate prediction mode at each point or multiple points in the template is determined, and then the first candidate prediction mode is determined according to the third cost of each point or multiple points in the template.
  • the third cost of the prediction model with respect to the template For example, the sum of the third costs of the first candidate prediction mode at the pixel points in the template is determined as the third cost of the first candidate prediction mode with respect to the template.
  • the third cost of the first candidate prediction mode with respect to the template can be determined by referring to the above formulas (1) and (2), specifically, replacing wTemplateValue[x][y] in the above formula (1) with pixels The weight value corresponding to the point (x, y) on the second template weight.
  • the third cost corresponding to each first candidate prediction mode among the M first candidate prediction modes can be determined, and then, the following S202-B11-4 is performed.
  • the first candidate prediction mode with the smallest third cost among the M first candidate prediction modes is determined as the first improved prediction mode corresponding to the jth initial prediction mode.
  • the above embodiment is described by taking the determination of the first improved prediction mode corresponding to the j-th initial prediction mode among the K initial prediction modes as an example. Referring to the above description, it is possible to determine the first improved prediction mode corresponding to each of the K initial prediction modes. The first improved prediction mode, and then K first improved prediction modes under the i-th candidate weight derivation mode are obtained.
  • use the i-th candidate weight derivation mode to derive the second template weight use K first improved prediction modes to predict the template, obtain K predicted values, and use the second template weight to weight the K predicted values, A second predicted value of the template is obtained, and a first cost corresponding to the i-th weight derivation mode is determined according to the second predicted value of the template and the reconstructed value of the template.
  • the determination of the first cost corresponding to the i-th candidate weight derivation mode is taken as an example.
  • the above-mentioned determination of the first cost corresponding to the i-th weight derivation mode can be used in this way, the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes is determined, and then the weight derivation mode of the current block is determined according to the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes. model.
  • the candidate weight derivation mode with the smallest cost among the N candidate weight derivation modes is determined as the weight derivation mode of the current block.
  • the N candidate weight derivation modes are sorted to obtain a candidate weight derivation mode list; according to the weight derivation mode index in the candidate weight derivation mode list, determine the current block Weight export mode.
  • K first improved prediction modes corresponding to each candidate weight derivation mode are determined.
  • the derivation mode according to the weight of the current block in the above S203-A includes the following steps:
  • each candidate weight derivation mode is determined Corresponding K first improved prediction modes.
  • the K first improved prediction modes corresponding to the weight derivation mode of the current block can be determined as K adjusted prediction modes, and then the weight of the current block can be used
  • the derived mode and the K adjusted prediction modes predict the current block to obtain the predicted value of the current block.
  • K first improved prediction modes corresponding to each candidate weight derivation mode are determined, and then according to each candidate weight derivation mode and its For the K first improved prediction modes, the template is predicted, and the first cost is determined. Based on the first cost, the weight derivation mode of the current block can be more accurately determined from the N candidate weight derivation modes.
  • the encoder in addition to using the above method 1 or method 2 to determine the weight derivation mode and K adjusted prediction modes of the current block, the encoder can also determine the weight derivation mode and K of the current block through the following method 3 an adjusted forecast model.
  • each candidate weight derivation mode is not used to improve the initial prediction mode, but simple P sub-templates are used, such as the current All templates, left templates, and upper templates of the block improve the initial prediction mode, thereby reducing the amount of calculation for improving the initial prediction mode.
  • determining the first cost corresponding to the i-th candidate weight derivation mode includes the following steps from S202-B21 to S202-B23:
  • the encoder uses P sub-templates to improve K initial prediction modes, and obtains a second improved prediction mode for each of the K initial prediction modes with respect to the P sub-templates.
  • the second improvement of each initial prediction mode from the K initial prediction modes with respect to the P sub-templates K second improved prediction modes corresponding to the candidate weight derivation mode are obtained, and then the template of the current block is predicted according to the candidate weight derivation mode and its corresponding K second prediction modes, and the candidate weight derivation mode is determined.
  • the first cost corresponding to the mode is corresponding to the mode.
  • the weight derivation mode of the current block is determined according to the first cost corresponding to each weight derivation mode in the N candidate weight derivation modes.
  • the candidate prediction modes are no longer used alone to improve the K initial prediction modes, thereby reducing the complexity of improving the initial prediction modes.
  • the P sub-templates include at least one of the left template TM_L, the upper template TM_A and the entire template TM_AL of the current block, wherein the entire template TM_AL includes the left template TM_L and the upper template TM_A.
  • the encoding end determines the second improved prediction mode of each of the K initial prediction modes with respect to the P sub-templates.
  • the j-th initial prediction mode is described with regard to the second improved prediction mode of the P sub-templates as an example.
  • determining the second improved prediction mode of the jth initial prediction mode with respect to the P sub-templates includes: determining the first Determine the second improved prediction mode of the j initial prediction mode with respect to the left template TM_L, determine the second improved prediction mode of the jth initial prediction mode with respect to the upper template TM_A, and determine the jth initial prediction mode with respect to the entire template TM_AL 2. Improve the prediction mode. That is to say, in this step, the encoder determines the second improved prediction mode of the j-th initial prediction mode with respect to each sub-template in the P sub-templates.
  • the embodiment of the present application does not limit the specific manner of determining the second improved prediction mode of the j-th initial prediction mode with respect to the P sub-templates.
  • S202-B21 includes the following steps from S202-B21-1 to S202-B21-3:
  • the encoding end determines the M first candidate prediction modes corresponding to the jth initial prediction mode, including the following two situations:
  • the aforementioned searching according to the jth initial prediction mode to obtain M pieces of first motion information includes: performing a search according to the ith initial prediction mode to obtain M offsets; according to the M offsets , to obtain M pieces of first motion information.
  • the encoding end After determining the M first candidate prediction modes corresponding to the j-th initial prediction mode according to the method of the above case 1 or case 2, the encoding end determines that the M first candidate prediction modes are respectively related to each sub-template in the P sub-templates The corresponding cost, and then determine the second improved prediction mode corresponding to each sub-template of the j-th initial prediction mode with respect to each of the P sub-templates according to the cost. For ease of description, the cost corresponding to the first candidate prediction mode at this time is recorded as the fourth cost.
  • the fourth cost of the first candidate prediction mode with respect to the P th sub template can be determined.
  • the jth initial prediction mode corresponds to three first candidate prediction modes, which are respectively denoted as the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3, assuming that the P sub-templates include the left Template TM_L, upper template TM_A, and all templates TM_AL.
  • the left template TM_L use the first candidate prediction mode 1, the first candidate prediction mode 2 and the first candidate prediction mode 3 to predict the left template TM_L respectively, and the corresponding prediction values are prediction value 1, prediction Value 2 and predicted value 3.
  • the fourth cost of the first candidate prediction mode 1 with respect to the left template TM_L is determined, and according to the prediction value 2 and the reconstruction value of the left template TM_L, the first The fourth cost of the candidate prediction mode 2 with respect to the left template TM_L determines the fourth cost of the first candidate prediction mode 3 with respect to the left template TM_L according to the prediction value 3 and the reconstructed value of the left template TM_L.
  • the upper template TM_A use the first candidate prediction mode 1, the first candidate prediction mode 2 and the first candidate prediction mode 3 to predict the upper template TM_A respectively, and the corresponding predicted values are predicted values 4, Predicted value 5 and predicted value 6.
  • the fourth cost of the first candidate prediction mode 1 with respect to the upper template TM_A is determined, and according to the predicted value 5 and the reconstruction value of the upper template TM_A, the first The fourth cost of the upper template TM_A for the candidate prediction mode 2 determines the fourth cost of the upper template TM_A for the first candidate prediction mode 3 according to the prediction value 6 and the reconstructed value of the upper template TM_A.
  • the fourth cost of each first candidate prediction mode with respect to each of the P sub-templates is determined, and then, the following S202-B21 is performed -3.
  • the first candidate prediction mode corresponding to the smallest fourth cost on the p-th sub-template is determined as the second improved prediction mode of the j-th initial prediction mode on the p-th sub-template.
  • the j-th initial prediction mode corresponds to three first candidate prediction modes, which are respectively denoted as the first candidate prediction mode 1, the first candidate prediction mode 2, and the first candidate prediction mode 3, assuming that the P sub-templates include Left template TM_L, upper template TM_A, and all templates TM_AL.
  • the fourth cost of the first candidate prediction mode 1 with respect to the upper template TM_A is less than the fourth cost of the first candidate prediction mode 2 with respect to the upper template TM_A
  • the first candidate prediction mode 2 with respect to the upper template TM_A The fourth cost of is less than the fourth cost of the first candidate prediction mode 3 with respect to the upper template TM_A, so that the first candidate prediction mode 1 can be determined as the second improved prediction mode of the jth initial prediction mode with respect to the left template TM_L.
  • the fourth cost of the first candidate prediction mode 2 with respect to the upper template TM_A is smaller than the fourth cost of the first candidate prediction mode 1 with respect to the upper template TM_A
  • the first candidate prediction mode 1 with respect to the upper template TM_A The fourth cost of the side template TM_A is smaller than the fourth cost of the first candidate prediction mode 3 with respect to the upper template TM_A, so that the first candidate prediction mode 2 can be determined as the second improvement of the jth initial prediction mode with respect to the upper template TM_A predictive mode.
  • the fourth cost of the first candidate prediction mode 2 for all templates TM_AL is less than the fourth cost of the first candidate prediction mode 1 for all templates TM_AL
  • the first candidate prediction mode 1 for all templates TM_AL The fourth cost is smaller than the fourth cost of the first candidate prediction mode 3 with respect to all templates TM_AL, so that the first candidate prediction mode 2 can be determined as the second improved prediction mode of the jth initial prediction mode with respect to all templates TM_AL.
  • the second improved prediction mode for each of the P sub-templates for each initial prediction mode may be determined.
  • the two initial prediction modes are respectively recorded as the first prediction mode and the second prediction mode
  • the P sub-templates are the left template TM_L, the upper template TM_A and all templates TM_AL
  • each initial prediction mode from the K initial prediction modes is related to the P sub-templates Among the second improved prediction modes of the template, K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined.
  • the sub-templates corresponding to the i-th candidate weight derivation mode are the upper template TM_A and all templates TM_AL.
  • the improved prediction mode is the second improved prediction mode 12
  • the second improved prediction mode of the second prediction mode for all templates TM_AL is the second improved prediction mode 23
  • two second second prediction modes corresponding to the i-th candidate weight derivation mode can be determined.
  • the improved prediction modes are respectively the second improved prediction mode 12 and the second improved prediction mode 23 .
  • the encoder determines the division angle corresponding to the i-th candidate weight derivation mode, and according to the correspondence between the division angle and the template of the prediction mode (refer to the above Table 5 for details), from the K Among the second improved prediction modes of the P sub-templates in the initial prediction mode, K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined.
  • K second improved prediction modes corresponding to the i-th candidate weight mode are determined, and then the following steps of S202-B23 are performed.
  • the first cost corresponding to the i weight derivation mode For example, determine the template weight according to the i-th candidate weight derivation mode, use K second prediction improvement modes to predict the template of the current block respectively, obtain K predicted values, and use the determined template weight to weight the K predicted values processing to obtain the template prediction value corresponding to the i-th candidate weight derivation mode, and then determine the first cost corresponding to the i-th candidate weight derivation mode according to the template prediction value and the template reconstruction value.
  • the determination of the first cost corresponding to the i-th candidate weight derivation mode is taken as an example.
  • the above-mentioned determination of the first cost corresponding to the i-th weight derivation mode can be used in this way, the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes is determined, and then the weight derivation mode of the current block is determined according to the first cost corresponding to each candidate weight derivation mode in the N candidate weight derivation modes. model.
  • the candidate weight derivation mode with the smallest cost among the N candidate weight derivation modes is determined as the weight derivation mode of the current block.
  • sort the N candidate weight derivation modes to obtain a candidate weight derivation mode list; determine the weight derivation mode of the current block according to the candidate weight derivation mode list.
  • P sub-templates are used to improve the K initial prediction modes to obtain a second improved prediction mode for each of the P sub-templates for each of the K initial prediction modes.
  • the K initial prediction modes are no longer improved according to different candidate weight derivation modes, but are determined uniformly from the above
  • the K initial prediction modes of each initial prediction mode in the second improved prediction mode of each sub-template in each of the P sub-templates, the K second improved prediction modes corresponding to each candidate weight derivation mode are queried, thereby reducing the need for the initial
  • the complexity of the prediction mode is improved, and the speed of calculating the first cost corresponding to each candidate weight derivation mode is increased, so that the weight derivation mode of the current block can be quickly determined from the N candidate weight derivation modes.
  • K second improved prediction modes corresponding to each candidate weight derivation mode are determined.
  • the ways to determine the K adjusted prediction modes include but not limited to the following:
  • K second improved prediction modes corresponding to the weight derivation mode of the current block are determined as K adjusted prediction modes.
  • K second improved prediction modes corresponding to each candidate weight derivation mode are determined, so that according to the first After the weight derivation mode of the current block is determined, K second improved prediction modes corresponding to the weight derivation mode of the current block may be directly determined as K adjusted prediction modes.
  • the K second improved prediction modes corresponding to the weight derivation mode of the current block are improved to obtain the K adjusted prediction modes.
  • the above S203 -A includes the following steps:
  • S203-A3 includes but is not limited to the following examples:
  • Example 1 Determine at least one prediction mode related to the weight derivation mode of the current block.
  • the at least one prediction mode may be a prediction mode whose prediction angle is parallel or approximately parallel to the boundary line of the weight derivation mode of the current block or a prediction mode whose prediction angle is parallel to the current block’s weight derivation mode.
  • the weights of the blocks are derived from the mode's vertical or nearly vertical prediction modes.
  • the encoder can obtain K adjusted prediction modes according to the cost corresponding to the new prediction mode combination and the costs corresponding to the K second improved prediction modes.
  • the The K prediction modes included in the new prediction mode combination 1 are determined as K adjusted prediction modes, and the new prediction mode combination 1 may include at least one initial prediction mode, or may not include the initial prediction mode.
  • Example 2 through the following steps from S203-A31 to S203-A34, K second prediction modes are improved to obtain K adjusted prediction modes:
  • the specific process of determining the weight of the first template can refer to the specific description of the derivation embodiment of the template weight below, which will not be repeated here.
  • the above S203-A32 and the above S203-A31 are executed in no order, that is, the above S203-A32 can be executed after the above S203-A31, or can be executed before the above S203-A31, or it can be executed together with the above S203-A31.
  • A31 is executed synchronously, which is not limited in this embodiment of the present application.
  • the encoder uses the weight derivation mode of the current block to improve each of the K second improved prediction modes, and obtains the corresponding The adjusted forecasting modes, and then get K adjusted forecasting modes.
  • the process of determining the adjusted prediction mode corresponding to each second improved prediction mode in the K second improved prediction modes is the same.
  • the embodiment of the present application determines the k-th II.
  • the adjusted prediction mode corresponding to the improved prediction mode is described as an example.
  • the encoder first determines P second candidate prediction modes corresponding to the kth second improved prediction mode.
  • the manner in which the encoding end determines the P second candidate prediction modes corresponding to the k-th second improved prediction mode includes the following two situations:
  • P intra-frame prediction modes similar to the k-th second improved prediction mode are determined as P second candidate prediction modes. For example, if the kth second improved prediction mode is an angle prediction mode, then the P angle prediction modes that are similar (or similar) to the angle of the kth second improved prediction mode are determined to be corresponding to the kth second improved prediction mode The P second candidate prediction modes of .
  • the k-th second improved prediction mode is an inter-frame prediction mode
  • search according to the k-th second improved prediction mode to obtain P pieces of second motion information, and determine the P pieces of second motion information based on the P pieces of second motion information.
  • P second candidate prediction modes corresponding to the k second improved prediction modes For example, for the kth second improved prediction mode, its motion vector is (xInit, yInit), set a search range such as from xInit-sR to xInit+sR in the horizontal direction, and from yInit-sR to yInit+sR in the vertical direction A rectangular area, where sR can be 2, 4, 8, etc.
  • Each motion vector in the rectangular area can be combined with other information of the kth second improved prediction mode, such as reference frame index and prediction list flag, etc. to determine a motion information, thereby determining a prediction mode, so that the first P second candidate prediction modes corresponding to the k second improved prediction modes.
  • other information of the kth second improved prediction mode such as reference frame index and prediction list flag, etc.
  • the above searching according to the kth second improved prediction mode to obtain P pieces of second motion information includes: performing search according to the kth second improved prediction mode to obtain P offsets; According to the P offsets, P pieces of second motion information are obtained.
  • the second candidate prediction mode for each of the P second candidate prediction modes, use the second candidate prediction mode to predict the template of the current block to obtain a prediction value, and then, according to the first template weight and The prediction value determines a fifth cost of the first candidate prediction mode with respect to the template.
  • use the second candidate prediction mode to predict the pixel to obtain a predicted value of the pixel, determine the template weight corresponding to the pixel in the first template weight, and set the The product of the predicted value of the pixel and the template weight determines the predicted value of the second candidate prediction mode at the pixel.
  • the predicted value of the second candidate prediction mode at each pixel point in the template can be obtained, and these predicted values form the template predicted value corresponding to the second candidate prediction mode. According to the template predicted value and the reconstructed value of the template, A fifth cost corresponding to the second candidate prediction mode is determined.
  • the fifth cost corresponding to each of the P second candidate prediction modes can be determined, and then, the following S203-A14 is performed.
  • the second candidate prediction mode with the fifth smallest cost among the P second candidate prediction modes is determined as the adjusted prediction mode corresponding to the kth second improved prediction mode.
  • the above embodiment is described by taking the determination of the adjusted prediction mode corresponding to the k-th second improved prediction mode among the K second improved prediction modes as an example. Referring to the above description, each of the K second improved prediction modes can be determined. The adjusted prediction mode corresponding to the second improved prediction mode, and then K adjusted prediction modes are obtained.
  • the encoder first uses P sub-templates to improve the K initial prediction modes to obtain K second improved prediction modes corresponding to the weight derivation mode of the current block, and then uses the weight derivation mode of the current block to improve the current block
  • the K second improved prediction modes corresponding to the weight derivation mode of the weight are improved to obtain accurate K adjusted prediction modes, so that when the current block is predicted using the K adjusted prediction modes and the weight derivation mode of the current block, it can be Further improve the accuracy of forecasting.
  • the weight derivation mode of the current block and K adjusted prediction modes can be determined, and then the current block can be predicted by using the weight derivation mode of the current block and the K adjusted prediction modes to obtain the current The predicted value of the block.
  • the ways of determining the prediction value of the current block include but are not limited to the following:
  • the above S203-B includes the following steps:
  • S203-B13 Determine the prediction value of the current block according to the K prediction values and the weight of the prediction value.
  • the weight derivation mode of the current block is used to determine the weight of the prediction value used by the current block.
  • the weight derivation mode of the current block may be a mode of deriving the weight of the prediction value.
  • each weight derivation mode can derive a weight matrix of predictive values; for blocks of the same size, the weight matrices of predictive values derived from different weight derivation modes can be different.
  • the encoder when determining the prediction value of the current block based on the K adjusted prediction modes and the weights of the prediction values, the encoder may first determine K predictions based on the K adjusted prediction modes value; then use the weight of the predicted value to perform weighted average calculation on the K predicted values, and finally the predicted value of the current block can be obtained.
  • determining the weight of the first template and the weight of the predicted value according to the weight derivation mode of the current block are two independent processes without interfering with each other.
  • the present application may also use the second method below to determine the predicted value of the current block.
  • S203-B includes the following steps:
  • S204-B23 Determine the predicted value of the current block according to the K predicted values and the weights of the K predicted values.
  • the weight of the pixels in the merged area is derived according to the weight derivation mode of the current block, and the merged area includes the current block and the template area of the current block, and then the weight corresponding to the current block in the merged area is The weight is determined as the weight of the predicted value, and the weight corresponding to the template region in the merged region is determined as the first template weight. That is to say, in the second method, the template area and the current block are taken as a whole area, and the first template weight and the weight of the predicted value can be derived in one step, which reduces the steps of weight derivation and improves the prediction effect.
  • the above prediction process is performed in units of pixels, and the corresponding weights are also weights corresponding to pixels.
  • each of the K adjusted prediction modes is used to predict a certain pixel A in the current block, and the K adjusted prediction modes with respect to the pixel A are obtained.
  • the weights of the K predictive values of pixel A are determined according to the weight derivation mode of the current block, and the K predictive values are weighted to obtain the predictive value of pixel A.
  • the first predicted value and the second predicted value are weighted according to the weight of the predicted value corresponding to pixel A to obtain the predicted value of pixel A.
  • both the first prediction mode and the second prediction mode are intra-frame prediction modes
  • the first intra-frame prediction mode is used for prediction to obtain the first predicted value
  • the second The intra prediction mode performs prediction to obtain the second prediction value, and weights the first prediction value and the second prediction value according to the weight of the prediction value to obtain the prediction value of the current block.
  • the first intra-frame prediction mode is used to predict the pixel point A to obtain the first predicted value of the pixel point A
  • the second intra-frame prediction mode is used to predict the pixel point A to obtain the second predicted value of the pixel point A
  • the first predicted value and the second predicted value are weighted to obtain the predicted value of the pixel point A.
  • the above S203-B determines the current block according to the K adjusted prediction modes and the weight derivation mode of the current block.
  • the predicted value of a block includes the following steps:
  • S203-B35 Determine the predicted value of the current block according to the jth predicted value, the K-1 predicted values and the weight of the predicted values.
  • the intra prediction mode is used for prediction
  • the first prediction value is obtained
  • the inter prediction mode is used for prediction , to obtain the second prediction value, and weight the first prediction value and the second prediction value according to the weight of the prediction value to obtain the prediction value of the current block.
  • the intra prediction mode is used to predict each point in the current block to obtain the predicted value of each point in the current block, and the predicted value of each point in the current block constitutes the first predicted value of the current block.
  • the inter-frame prediction mode determine a piece of motion information, determine the best matching block of the current block according to the motion information, and determine the best matching block as the second prediction value of the current block. For the weight of the prediction value of each pixel in the current block, a point-by-point weighting operation is performed on the first prediction value and the second prediction value of the current block to obtain the prediction value of the current block. For example, for pixel A in the current block, according to the weight of the prediction value of pixel A, the first prediction value corresponding to pixel A in the first prediction value of the current block is compared with the pixel in the second prediction value of the current block The second predicted value corresponding to point A is weighted to obtain the predicted value of pixel point A.
  • the weights of the predicted values corresponding to any two adjusted prediction modes among the K adjusted prediction modes can be determined according to the weight derivation mode of the current block, and the weights of the predicted values corresponding to any two adjusted prediction modes among the K adjusted prediction modes can be determined.
  • the weights of the predicted values corresponding to the K adjusted prediction modes can be determined according to the preset weight ratio, Assuming that the weight of the predicted value corresponding to the third prediction mode accounts for 1/4 of the total weight of the predicted value in the entire medium, it can be determined that the total weight of the predicted value of the third prediction mode is 2, and the remaining 3 of the total weight of the predicted value /4 is assigned to the first prediction mode and the second prediction mode.
  • the weight of the prediction value corresponding to the first prediction mode is determined to be (3/4)*3, and the weight of the prediction value corresponding to the second prediction mode
  • the weight of the corresponding predicted value is (3/4)*5 for the predicted value of the first prediction mode.
  • the encoding end determines the predicted value of the current block according to the above method, and then determines the difference between the current block and the predicted value as the residual value of the current block, transforms the residual value of the current block, obtains the transformation coefficient, and transforms
  • the coefficients are quantized to obtain quantized coefficients, and the quantized coefficients are encoded to obtain a code stream.
  • the encoder determines K initial prediction modes; determines the weight derivation mode of the current block according to the K initial prediction modes; and determines the prediction value of the current block according to the weight derivation mode of the current block. That is, the present application determines the weight derivation mode of the current block through K initial prediction modes, thereby enriching the determination method of the weight derivation mode.
  • the encoder may not transmit the index of the weight derivation mode in the code stream, thereby saving codewords and reducing the encoding cost.
  • sequence numbers of the above-mentioned processes do not mean the order of execution, and the order of execution of the processes should be determined by their functions and internal logic, and should not be used in this application.
  • the implementation of the examples constitutes no limitation.
  • the term "and/or" is only an association relationship describing associated objects, indicating that there may be three relationships. Specifically, A and/or B may mean: A exists alone, A and B exist simultaneously, and B exists alone.
  • the character "/" in this application generally indicates that the contextual objects are an "or" relationship.
  • FIG. 20 is a schematic block diagram of a prediction device provided by an embodiment of the present application, and the prediction device 10 is applied to the above-mentioned video decoder.
  • the forecasting device 10 includes:
  • the decoding unit 11 is used to decode the code stream and determine K initial prediction modes, where K is a positive integer greater than 1;
  • a determining unit 12 configured to determine a weight derivation mode of the current block according to the K initial prediction modes
  • the prediction unit 13 is configured to determine a prediction value of the current block according to a weight derivation mode of the current block.
  • the prediction unit 13 is specifically configured to determine K adjusted prediction modes according to the weight derivation mode of the current block; according to the K adjusted prediction modes and the weight derivation mode of the current block, A prediction value for the current block is determined.
  • the determining unit 12 is specifically configured to determine N candidate weight derivation modes, where N is a positive integer greater than 1; for the i-th candidate weight derivation mode among the N candidate weight derivation modes, According to the i-th candidate weight derivation mode and the K initial prediction modes, determine the first cost corresponding to the i-th candidate weight derivation mode, where i is a positive integer from 1 to N; according to the The first costs respectively corresponding to the N candidate weight derivation modes determine the weight derivation mode of the current block.
  • the determining unit 12 is specifically configured to use the ith candidate weight derivation mode and the K initial prediction modes to predict the template of the current block, and obtain the first predicted value of the template ; Determine a first cost corresponding to the ith weight derivation mode according to the first predicted value of the template and the reconstructed value of the template.
  • the determining unit 12 is specifically configured to improve the K initial prediction modes according to the weight derivation mode of the current block, to obtain the K adjusted prediction modes.
  • the determining unit 12 is specifically configured to determine the first template weight according to the weight derivation mode of the current block; and determine the first template weight for the jth initial prediction mode among the K initial prediction modes M first candidate prediction modes corresponding to the j initial prediction modes, where M is a positive integer greater than 1, and j is a positive integer from 1 to K; according to the first template weight, it is determined to use the M The second costs corresponding to the first candidate prediction modes respectively predicting the template; according to the second costs corresponding to the M first candidate prediction modes, determine the adjustment corresponding to the jth initial prediction mode prediction model.
  • the determining unit 12 is specifically configured to use the first candidate prediction mode with the smallest second cost among the M first candidate prediction modes to determine the adjusted value corresponding to the jth initial prediction mode predictive mode.
  • the determining unit 12 is specifically configured to improve the K initial prediction modes according to the i-th candidate weight derivation mode to obtain the K-th candidate weight derivation modes corresponding to the i-th candidate weight derivation mode.
  • An improved prediction mode according to the i-th candidate weight derivation mode and the K first improved prediction modes, determine a first cost corresponding to the i-th candidate weight derivation mode.
  • the determining unit 12 is specifically configured to determine the second template weight according to the i-th candidate weight derivation mode; for the j-th initial prediction mode among the K initial prediction modes, determine the The M first candidate prediction modes corresponding to the jth initial prediction mode, the M is a positive integer, and the j is a positive integer from 1 to K; according to the second template weight, it is determined to use the M first A candidate prediction mode corresponds to a third cost when predicting the template; according to the third costs corresponding to the M first candidate prediction modes, determine the first improvement corresponding to the jth initial prediction mode predictive mode.
  • the determining unit 12 is specifically configured to determine the first candidate prediction mode with the smallest third cost among the M first candidate prediction modes, and determine the first candidate prediction mode corresponding to the jth initial prediction mode. Improve predictive models.
  • the determining unit 12 is specifically configured to use the ith candidate weight derivation mode and the K first improved prediction modes to predict the template of the current block, and obtain the second A predicted value: determining a first cost corresponding to the i-th weight derivation mode according to the second predicted value of the template and the reconstructed value of the template.
  • the determining unit 12 is specifically configured to determine the K first improved prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.
  • the determining unit 12 is specifically configured to, for the jth initial prediction mode among the K initial prediction modes, determine the second improved prediction mode of the jth initial prediction mode with respect to the P sub-templates,
  • the P is a positive integer
  • the j is a positive integer from 1 to K; according to the i-th candidate weight derivation mode, from the K initial prediction modes with respect to the second improved prediction of the P sub-templates respectively mode, determine K second improved prediction modes corresponding to the i-th candidate weight derivation mode; determine the i-th The first cost corresponding to candidate weight derivation modes.
  • the determining unit 12 is specifically configured to determine the division angle corresponding to the ith candidate weight derivation mode; according to the correspondence between the division angle and the template of the prediction mode, from the K initial Among the second improved prediction modes of the P sub-templates, K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined.
  • the P sub-templates include at least one of the left template, the upper template and all templates of the current block.
  • the determining unit 12 is specifically configured to determine M first candidate prediction modes corresponding to the jth initial prediction mode, where M is a positive integer greater than 1; sub-templates, determine the corresponding fourth cost when using the M first candidate prediction modes to predict the p-th sub-template respectively, and the p is a positive integer from 1 to P; according to the M-th sub-templates A candidate prediction mode is respectively related to the fourth cost of the p-th sub-template, and a second improved prediction mode of the j-th initial prediction mode with respect to the p-th sub-template is determined.
  • the determining unit 12 is specifically configured to determine, among the M first candidate prediction modes, the first candidate prediction mode corresponding to the smallest fourth cost of the p-th sub-template as the j-th The second improved prediction mode of the initial prediction mode for the p-th sub-template.
  • the determining unit 12 is specifically configured to, for any first candidate prediction mode of the M first candidate prediction modes, use the first candidate prediction mode to predict the p-th sub-template, Obtaining the predicted value of the pth sub-template; according to the predicted value of the pth sub-template and the reconstructed value of the pth sub-template, determining the fourth cost.
  • the determining unit 12 is specifically configured to use the ith candidate weight derivation mode and the K second improved prediction modes to predict the template of the current block, and obtain the third A predicted value: determining a first cost corresponding to the i-th weight derivation mode according to the third predicted value of the template and the reconstructed value of the template.
  • the determining unit 12 is specifically configured to determine the K adjusted prediction modes according to the K second improved prediction modes corresponding to the weight derivation mode of the current block.
  • the determining unit 12 is specifically configured to determine the K second improved prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.
  • the determining unit 12 is specifically configured to improve the K second improved prediction modes according to the weight derivation mode of the current block, to obtain the K adjusted prediction modes.
  • the determining unit 12 is specifically configured to determine the first template weight according to the weight derivation mode of the current block; for the kth second improved prediction mode among the K second improved prediction modes, Determine P second candidate prediction modes corresponding to the kth second improved prediction mode, where P is a positive integer greater than 1, and k is a positive integer from 1 to K; according to the weight of the first template , determine the corresponding fifth cost when using the P second candidate prediction modes to predict the template respectively; according to the fifth costs corresponding to the P second candidate prediction modes, determine the kth The adjusted prediction mode corresponding to the second improved prediction mode.
  • the determining unit 12 is specifically configured to determine the second candidate prediction mode with the smallest fifth cost among the P second candidate prediction modes as the k-th second improved prediction mode corresponding to The adjusted forecast model.
  • the determining unit 12 is specifically configured to determine M intra-frame prediction modes similar to the j-th initial prediction mode as The M first candidate prediction modes.
  • the determining unit 12 is specifically configured to search according to the j-th initial prediction mode to obtain M pieces of first motion information; according to The M pieces of first motion information are obtained from the M pieces of first candidate prediction modes.
  • the determining unit 12 is specifically configured to search according to the ith initial prediction mode to obtain M offsets; according to the M offsets, obtain the M first motion information .
  • the determining unit 12 is specifically configured to determine P intra-frame prediction modes similar to the k-th second improved prediction mode as the P second candidate prediction modes.
  • the determining unit 12 is specifically configured to search according to the kth second improved prediction mode to obtain P pieces of second motion information; according to the P pieces of second motion information, to obtain the P pieces of The second candidate prediction mode.
  • the determining unit 12 is specifically configured to search according to the kth second improved prediction mode to obtain P offsets; according to the P offsets, obtain P second motion information .
  • the determining unit 12 is specifically configured to determine, among the N candidate weight derivation modes, the candidate weight derivation mode with the smallest first cost as the weight derivation mode of the current block.
  • the determining unit 12 is specifically configured to sort the N candidate weight derivation modes according to the first costs respectively corresponding to the N candidate weight derivation modes, to obtain a list of candidate weight derivation modes;
  • the code stream is used to obtain a weight derivation mode index; and the weight derivation mode index corresponding to the candidate weight derivation mode in the candidate weight derivation mode list is determined as the weight derivation mode of the current block.
  • the decoding unit 11 is specifically configured to determine a candidate prediction mode list, the candidate prediction mode list includes at least two candidate prediction modes; decode the code stream to obtain a prediction mode index; according to the The prediction mode index is used to determine the K initial prediction modes from the candidate prediction mode list.
  • the determination unit 12 is specifically configured to determine the angle index and the distance index according to the weight derivation mode; determine the template weight according to the angle index, the distance index and the size of the template; if the weight derivation mode When it is the weight derivation mode of the current block, the template weight is the first template weight, and if the weight derivation mode is the ith candidate weight derivation mode, the template weight is the first template weight Two template weights.
  • the determining unit 12 is specifically configured to determine a first parameter of a pixel in the template according to the angle index, the distance index, and the size of the template, and the first parameter is used to determine Weight: determining the weight of the pixel in the template according to the first parameter of the pixel in the template; determining the weight of the template according to the weight of the pixel in the template.
  • the prediction unit 13 is specifically configured to determine K predictive values according to the K adjusted prediction modes; determine the weights of the K predictive values according to the weight derivation mode of the current block; The weights of the K prediction modes and the K prediction values determine the prediction value of the current block.
  • the determining unit 12 is specifically configured to determine the merged area formed by the current block and the template according to the angle index, the distance index, the size of the template, and the size of the current block
  • the prediction unit 12 is specifically configured to determine K predictive values according to the K adjusted prediction modes; determine weights of the K predictive values according to weights of pixels in the merged area; Determine the predicted value of the current block according to the K predicted values and the weights of the K predicted values.
  • the prediction unit 12 is specifically configured to determine the weights of the K prediction values according to the weights corresponding to the pixels of the current block in the merged area.
  • the template includes at least one of the upper decoded area, the left decoded area, and the upper left decoded area of the current block.
  • the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, details are not repeated here.
  • the device 10 shown in FIG. 20 can execute the prediction method at the decoding end of the embodiment of the present application, and the aforementioned and other operations and/or functions of each unit in the device 10 are to realize the above-mentioned prediction method at the decoding end and other methods. For the sake of brevity, the corresponding process will not be repeated here.
  • Fig. 21 is a schematic block diagram of a prediction device provided by an embodiment of the present application, and the prediction device is applied to the above encoder.
  • the prediction device 20 may include:
  • the first determining unit 21 is configured to determine K initial prediction modes, where K is a positive integer greater than 1;
  • the second determination unit 22 is configured to determine the weight derivation mode of the current block according to the K initial prediction modes
  • the prediction unit 23 is configured to determine a prediction value of the current block according to the weight derivation mode of the current block.
  • the prediction unit 23 is specifically configured to determine K adjusted prediction modes according to the weight derivation mode of the current block; according to the K adjusted prediction modes and the weight derivation mode of the current block, A prediction value for the current block is determined.
  • the second determination unit 22 is specifically configured to determine N candidate weight derivation modes, where N is a positive integer greater than 1; for the i-th candidate weight derivation mode in the N candidate weight derivation modes, mode, according to the i-th candidate weight derivation mode and the K initial prediction modes, determine the first cost corresponding to the i-th candidate weight derivation mode, where i is a positive integer from 1 to N; according to The first costs respectively corresponding to the N candidate weight derivation modes determine the weight derivation mode of the current block.
  • the second determination unit 22 is specifically configured to use the ith candidate weight derivation mode and the K initial prediction modes to predict the template of the current block, and obtain the first A predicted value: determining a first cost corresponding to the i-th weight derivation mode according to the first predicted value of the template and the reconstructed value of the template.
  • the second determining unit 22 is specifically configured to improve the K initial prediction modes according to the weight derivation mode of the current block to obtain the K adjusted prediction modes.
  • the second determination unit 22 is specifically configured to determine the first template weight according to the weight derivation mode of the current block; for the jth initial prediction mode among the K initial prediction modes, determine the M first candidate prediction modes corresponding to the jth initial prediction mode, where M is a positive integer greater than 1, and j is a positive integer from 1 to K; according to the weight of the first template, it is determined to use the The second costs corresponding to the M first candidate prediction modes when predicting the template respectively; according to the second costs corresponding to the M first candidate prediction modes, determine the jth initial prediction mode corresponding to The adjusted forecast model.
  • the second determination unit 22 is specifically configured to determine the first candidate prediction mode with the smallest second cost among the M first candidate prediction modes, and determine the first candidate prediction mode corresponding to the jth initial prediction mode Adjusted forecast mode.
  • the second determination unit 22 is specifically configured to improve the K initial prediction modes according to the i-th candidate weight derivation mode, and obtain K corresponding to the i-th candidate weight derivation mode. a first improved prediction mode; according to the i-th candidate weight derivation mode and the K first improved prediction modes, determine a first cost corresponding to the i-th candidate weight derivation mode.
  • the second determination unit 22 is specifically configured to determine the second template weight according to the i-th candidate weight derivation mode; for the j-th initial prediction mode among the K initial prediction modes, determine The M first candidate prediction modes corresponding to the jth initial prediction mode, the M is a positive integer, and the j is a positive integer from 1 to K; according to the weight of the second template, it is determined to use the M The third costs corresponding to the first candidate prediction modes respectively predicting the template; according to the third costs corresponding to the M first candidate prediction modes, determine the first corresponding to the jth initial prediction mode - Improved prediction mode.
  • the second determining unit 22 is specifically configured to determine the first candidate prediction mode with the smallest third cost among the M first candidate prediction modes, and determine the first candidate prediction mode corresponding to the jth initial prediction mode The first improved prediction model.
  • the second determining unit 22 is specifically configured to use the ith candidate weight derivation mode and the K first improved prediction modes to predict the template of the current block, and obtain the A second predicted value: determining a first cost corresponding to the i-th weight derivation mode according to the second predicted value of the template and the reconstructed value of the template.
  • the second determining unit 22 is specifically configured to determine the K first improved prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.
  • the second determination unit 22 is specifically configured to, for the jth initial prediction mode in the K initial prediction modes, determine the second improved prediction of the jth initial prediction mode with respect to the P sub-templates mode, the P is a positive integer, and the j is a positive integer from 1 to K; according to the i-th candidate weight derivation mode, from the K initial prediction modes for the second of the P sub-templates respectively In the improved prediction mode, determine K second improved prediction modes corresponding to the i-th candidate weight derivation mode; according to the i-th candidate weight derivation mode and the K second improved prediction modes, determine the The first cost corresponding to the ith candidate weight derivation mode.
  • the second determination unit 22 is specifically configured to determine the division angle corresponding to the ith candidate weight derivation mode; according to the correspondence between the division angle and the template of the prediction mode, from the K K second improved prediction modes corresponding to the i-th candidate weight derivation mode are determined from among the second improved prediction modes of the P sub-templates for each of the initial prediction modes.
  • the P sub-templates include at least one of the left template, the upper template and all templates of the current block.
  • the second determination unit 22 is specifically configured to determine M first candidate prediction modes corresponding to the jth initial prediction mode, where M is a positive integer greater than 1; For the p-th sub-template, determine the corresponding fourth cost when using the M first candidate prediction modes to predict the p-th sub-template respectively, where p is a positive integer from 1 to P; according to the M The first candidate prediction modes are respectively related to the fourth cost of the p-th sub-template, and the second improved prediction mode of the j-th initial prediction mode with respect to the p-th sub-template is determined.
  • the second determining unit 22 is specifically configured to determine the first candidate prediction mode corresponding to the smallest fourth cost of the p-th sub-template among the M first candidate prediction modes as the The j-th initial prediction mode is the second improved prediction mode for the p-th sub-template.
  • the second determining unit 22 is specifically configured to, for any first candidate prediction mode of the M first candidate prediction modes, use the first candidate prediction mode to perform the p-th sub-template Prediction, obtaining the predicted value of the p-th sub-template; according to the predicted value of the p-th sub-template and the reconstructed value of the p-th sub-template, determine the first candidate prediction mode with respect to the P-th sub-template Fourth price.
  • the second determining unit 22 is specifically configured to use the ith candidate weight derivation mode and the K second improved prediction modes to predict the template of the current block, and obtain the A third predicted value: determining a first cost corresponding to the i-th weight derivation mode according to the third predicted value of the template and the reconstructed value of the template.
  • the second determining unit 22 is specifically configured to determine the K adjusted prediction modes according to the K second improved prediction modes corresponding to the weight derivation mode of the current block.
  • the second determining unit 22 is specifically configured to determine the K second improved prediction modes corresponding to the weight derivation mode of the current block as the K adjusted prediction modes.
  • the second determining unit 22 is specifically configured to improve the K second improved prediction modes according to the weight derivation mode of the current block to obtain the K adjusted prediction modes.
  • the second determination unit 22 is specifically configured to determine the first template weight according to the weight derivation mode of the current block; for the kth second improved prediction in the K second improved prediction modes mode, determining P second candidate prediction modes corresponding to the kth second improved prediction mode, where P is a positive integer greater than 1, and k is a positive integer from 1 to K; according to the first
  • the template weight is used to determine the corresponding fifth cost when the P second candidate prediction modes are respectively used to predict the template; according to the fifth costs corresponding to the P second candidate prediction modes, the fifth cost is determined. Adjusted prediction modes corresponding to the k second improved prediction modes.
  • the second determining unit 22 is specifically configured to determine the second candidate prediction mode with the smallest fifth cost among the P second candidate prediction modes as the kth second improved prediction mode Mode corresponding to the adjusted prediction mode.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

本申请提供一种预测方法、装置、设备、系统、及存储介质，本申请通过通过K个初始预测模式，确定当前块的权重导出模式，进而丰富了权重导出模式的确定方式。另外，本申请中编码端在码流中可以不传输权重导出模式的索引，解码端也可以根据K个初始预测模式确定出当前块的权重导出模式，进而节约了码字，降低了编码代价。

Description

预测方法、装置、设备、系统、及存储介质 技术领域
本申请涉及视频编解码技术领域，尤其涉及一种预测方法、装置、设备、系统、及存储介质。
背景技术
数字视频技术可以并入多种视频装置中，例如数字电视、智能手机、计算机、电子阅读器或视频播放器等。随着视频技术的发展，视频数据所包括的数据量较大，为了便于视频数据的传输，视频装置执行视频压缩技术，以使视频数据更加有效的传输或存储。
由于视频中存在时间或空间冗余，通过预测可以消除或降低视频中的冗余，提高压缩效率。在一些情况下，使用多种预测模式对当前块进行预测，得到当前块的预测值，此时为了保持编解码两端的一致性，编码端需要将预测当前块所需的模式信息写入码流，占用较多的编码资源，提高了编码代价。
发明内容
本申请实施例提供了一种预测方法、装置、设备、系统、及存储介质，通过K个初始预测模式确定当前块的权重导出模式，进而降低编码代价。
第一方面，本申请提供了一种预测方法，应用于解码器，包括：
解码码流，确定K个初始预测模式，所述K为大于1的正整数；
根据所述K个初始预测模式，确定所述当前块的权重导出模式；
根据所述当前块的权重导出模式，确定所述当前块的预测值。
第二方面，本申请实施例提供一种预测方法，包括：
确定K个初始预测模式，所述K为大于1的正整数；
根据所述K个初始预测模式，确定当前块的权重导出模式；
根据所述当前块的权重导出模式，确定当前块的预测值。
第三方面，本申请提供了一种预测装置，用于执行上述第一方面或其各实现方式中的方法。具体地，该预测装置包括用于执行上述第一方面或其各实现方式中的方法的功能单元。
第四方面，本申请提供了一种预测装置，用于执行上述第二方面或其各实现方式中的方法。具体地，该预测装置包括用于执行上述第二方面或其各实现方式中的方法的功能单元。
第五方面，提供了一种视频解码器，包括处理器和存储器。该存储器用于存储计算机程序，该处理器用于调用并运行该存储器中存储的计算机程序，以执行上述第一方面或其各实现方式中的方法。
第六方面，提供了一种视频编码器，包括处理器和存储器。该存储器用于存储计算机程序，该处理器用于调用并运行该存储器中存储的计算机程序，以执行上述第二方面或其各实现方式中的方法。
第七方面，提供了一种视频编解码系统，包括视频编码器和视频解码器。视频解码器用于执行上述第一方面或其各实现方式中的方法，视频编码器用于执行上述第二方面或其各实现方式中的方法。
第八方面，提供了一种芯片，用于实现上述第一方面至第二方面中的任一方面或其各实现方式中的方法。具体地，该芯片包括：处理器，用于从存储器中调用并运行计算机程序，使得安装有该芯片的设备执行如上述第一方面至第二方面中的任一方面或其各实现方式中的方法。
第九方面，提供了一种计算机可读存储介质，用于存储计算机程序，该计算机程序使得计算机执行上述第一方面至第二方面中的任一方面或其各实现方式中的方法。
第十方面，提供了一种计算机程序产品，包括计算机程序指令，该计算机程序指令使得计算机执行上述第一方面至第二方面中的任一方面或其各实现方式中的方法。
第十一方面，提供了一种计算机程序，当其在计算机上运行时，使得计算机执行上述第一方面至第二方面中的任一方面或其各实现方式中的方法。
第十二方面，提供了一种码流，码流是基于上述第二方面的方法生成的。
基于以上技术方案，通过通过K个初始预测模式，确定当前块的权重导出模式，进而丰富了权重导出模式的确定方式。另外，本申请中编码端在码流中可以不传输权重导出模式的索引，解码端也可以根据K个初始预测模式确定出当前块的权重导出模式，进而节约了码字，降低了编码代价。
附图说明
图1为本申请实施例涉及的一种视频编解码系统的示意性框图；
图2是本申请实施例涉及的视频编码器的示意性框图；
图3是本申请实施例涉及的视频解码器的示意性框图；
图4为权重分配示意图；
图5为权重分配示意图；
图6A为帧间预测的示意图；
图6B为加权帧间预测的示意图；
图7A为帧内预测的示意图；
图7B为帧内预测的示意图；
图8A-8I为帧内预测的示意图；
图9为帧内预测模式的示意图；
图10为帧内预测模式的示意图；
图11为帧内预测模式的示意图；
图12为加权帧内预测的示意图；
图13为模板匹配示意图；
图14为本申请一实施例提供的预测方法流程示意图；
图15为使用两种预测模式对当前块进行预测时的示意图；
图16为一种模板划分示意图；
图17A为另一种模板划分示意图；
图17B为又一种模板划分示意图；
图18为一种模板尺寸示意图；
图19为本申请实一施例提供的预测方法流程示意图；
图20是本申请一实施例提供的预测装置的示意性框图；
图21是本申请一实施例提供的预测装置的示意性框图；
图22是本申请实施例提供的电子设备的示意性框图；
图23是本申请实施例提供的视频编解码系统的示意性框图。
具体实施方式
本申请可应用于图像编解码领域、视频编解码领域、硬件视频编解码领域、专用电路视频编解码领域、实时视频编解码领域等。例如，本申请的方案可结合至音视频编码标准(audio video coding standard，简称AVS)，例如，H.264/音视频编码(audio video coding，简称AVC)标准，H.265/高效视频编码(high efficiency video coding，简称HEVC)标准以及H.266/多功能视频编码(versatile video coding，简称VVC)标准。或者，本申请的方案可结合至其它专属或行业标准而操作，所述标准包含ITU-TH.261、ISO/IECMPEG-1Visual、ITU-TH.262或ISO/IECMPEG-2Visual、ITU-TH.263、ISO/IECMPEG-4Visual，ITU-TH.264(还称为ISO/IECMPEG-4AVC)，包含可分级视频编解码(SVC)及多视图视频编解码(MVC)扩展。应理解，本申请的技术不限于任何特定编解码标准或技术。
为了便于理解，首先结合图1对本申请实施例涉及的视频编解码系统进行介绍。
图1为本申请实施例涉及的一种视频编解码系统的示意性框图。需要说明的是，图1只是一种示例，本申请实施例的视频编解码系统包括但不限于图1所示。如图1所示，该视频编解码系统100包含编码设备110和解码设备120。其中编码设备用于对视频数据进行编码(可以理解成压缩)产生码流，并将码流传输给解码设备。解码设备对编码设备编码产生的码流进行解码，得到解码后的视频数据。
本申请实施例的编码设备110可以理解为具有视频编码功能的设备，解码设备120可以理解为具有视频解码功能的设备，即本申请实施例对编码设备110和解码设备120包括更广泛的装置，例如包含智能手机、台式计算机、移动计算装置、笔记本(例如，膝上型)计算机、平板计算机、机顶盒、电视、相机、显示装置、数字媒体播放器、视频游戏控制台、车载计算机等。
在一些实施例中，编码设备110可以经由信道130将编码后的视频数据(如码流)传输给解码设备120。信道130可以包括能够将编码后的视频数据从编码设备110传输到解码设备120的一个或多个媒体和/或装置。
在一个实例中，信道130包括使编码设备110能够实时地将编码后的视频数据直接发射到解码设备120的一个或多个通信媒体。在此实例中，编码设备110可根据通信标准来调制编码后的视频数据，且将调制后的视频数据发射到解码设备120。其中通信媒体包含无线通信媒体，例如射频频谱，可选的，通信媒体还可以包含有线通信媒体，例如一根或多根物理传输线。
在另一实例中，信道130包括存储介质，该存储介质可以存储编码设备110编码后的视频数据。存储介质包含多种本地存取式数据存储介质，例如光盘、DVD、快闪存储器等。在该实例中，解码设备120可从该存储介质中获取编码后的视频数据。
在另一实例中，信道130可包含存储服务器，该存储服务器可以存储编码设备110编码后的视频数据。在此实例中，解码设备120可以从该存储服务器中下载存储的编码后的视频数据。可选的，该存储服务器可以存储编码后的视频数据且可以将该编码后的视频数据发射到解码设备120，例如web服务器(例如，用于网站)、文件传送协议(FTP)服务器等。
一些实施例中，编码设备110包含视频编码器112及输出接口113。其中，输出接口113可以包含调制器/解调器(调制解调器)和/或发射器。
在一些实施例中，编码设备110除了包括视频编码器112和输入接口113外，还可以包括视频源111。
视频源111可包含视频采集装置(例如，视频相机)、视频存档、视频输入接口、计算机图形系统中的至少一个，其中，视频输入接口用于从视频内容提供者处接收视频数据，计算机图形系统用于产生视频数据。
视频编码器112对来自视频源111的视频数据进行编码，产生码流。视频数据可包括一个或多个图像(picture)或图像序列(sequence of pictures)。码流以比特流的形式包含了图像或图像序列的编码信息。编码信息可以包含编码图像数据及相关联数据。相关联数据可包含序列参数集(sequence parameter set，简称SPS)、图像参数集(picture parameter set，简称PPS)及其它语法结构。SPS可含有应用于一个或多个序列的参数。PPS可含有应用于一个或多个图像的参数。语法结构是指码流中以指定次序排列的零个或多个语法元素的集合。
视频编码器112经由输出接口113将编码后的视频数据直接传输到解码设备120。编码后的视频数据还可存储于存储介质或存储服务器上，以供解码设备120后续读取。
在一些实施例中，解码设备120包含输入接口121和视频解码器122。
在一些实施例中，解码设备120除包括输入接口121和视频解码器122外，还可以包括显示装置123。
其中，输入接口121包含接收器及/或调制解调器。输入接口121可通过信道130接收编码后的视频数据。
视频解码器122用于对编码后的视频数据进行解码，得到解码后的视频数据，并将解码后的视频数据传输至显示装置123。
显示装置123显示解码后的视频数据。显示装置123可与解码设备120整合或在解码设备120外部。显示装置123可包括多种显示装置，例如液晶显示器(LCD)、等离子体显示器、有机发光二极管(OLED)显示器或其它类型的显示装置。
此外，图1仅为实例，本申请实施例的技术方案不限于图1，例如本申请的技术还可以应用于单侧的视频编码或单侧的视频解码。
下面对本申请实施例涉及的视频编码框架进行介绍。
图2是本申请实施例涉及的视频编码器的示意性框图。应理解，该视频编码器200可用于对图像进行有损压缩(lossy compression)，也可用于对图像进行无损压缩(lossless compression)。该无损压缩可以是视觉无损压缩(visually lossless compression)，也可以是数学无损压缩(mathematically lossless compression)。
该视频编码器200可应用于亮度色度(YCbCr，YUV)格式的图像数据上。例如，YUV比例可以为4:2:0、4:2:2或者4:4:4，Y表示明亮度(Luma)，Cb(U)表示蓝色色度，Cr(V)表示红色色度，U和V表示为色度(Chroma)用于描述色彩及饱和度。例如，在颜色格式上，4:2:0表示每4个像素有4个亮度分量，2个色度分量(YYYYCbCr)，4:2:2表示每4个像素有4个亮度分量，4个色度分量(YYYYCbCrCbCr)，4:4:4表示全像素显示(YYYYCbCrCbCrCbCrCbCr)。
例如，该视频编码器200读取视频数据，针对视频数据中的每帧图像，将一帧图像划分成若干个编码树单元(coding tree unit，CTU)，在一些例子中，CTB可被称作“树型块”、“最大编码单元”(Largest Coding unit，简称LCU)或“编码树型块”(coding tree block，简称CTB)。每一个CTU可以与图像内的具有相等大小的像素块相关联。每一像素可对应一个亮度(luminance或luma)采样及两个色度(chrominance或chroma)采样。因此，每一个CTU可与一个亮度采样块及两个色度采样块相关联。一个CTU大小例如为128×128、64×64、32×32等。一个CTU又可以继续被划分成若干个编码单元(Coding Unit，CU)进行编码，CU可以为矩形块也可以为方形块。CU可以进一步划分为预测单元(prediction Unit，简称PU)和变换单元(transform unit，简称TU)，进而使得编码、预测、变换分离，处理的时候更灵活。在一种示例中，CTU以四叉树方式划分为CU，CU以四叉树方式划分为TU、PU。
视频编码器及视频解码器可支持各种PU大小。假定特定CU的大小为2N×2N，视频编码器及视频解码器可支持2N×2N或N×N的PU大小以用于帧内预测，且支持2N×2N、2N×N、N×2N、N×N或类似大小的对称PU以用于帧间预测。视频编码器及视频解码器还可支持2N×nU、2N×nD、nL×2N及nR×2N的不对称PU以用于帧间预测。
在一些实施例中，如图2所示，该视频编码器200可包括：预测单元210、残差单元220、变换/量化单元230、反变换/量化单元240、重建单元250、环路滤波单元260、解码图像缓存270和熵编码单元280。需要说明的是，视频编码器200可包含更多、更少或不同的功能组件。
可选的，在本申请中，当前块(current block)可以称为当前编码单元(CU)或当前预测单元(PU)等。预测块也可称为预测图像块或图像预测块，重建图像块也可称为重建块或图像重建图像块。
在一些实施例中，预测单元210包括帧间预测单元211和帧内估计单元212。由于视频的一个帧中的相邻像素之间存在很强的相关性，在视频编解码技术中使用帧内预测的方法消除相邻像素之间的空间冗余。由于视频中的相邻帧之间存在着很强的相似性，在视频编解码技术中使用帧间预测方法消除相邻帧之间的时间冗余，从而提高编码效率。
帧间预测单元211可用于帧间预测，帧间预测可以包括运动估计(motion estimation)和运动补偿(motion compensation)，可以参考不同帧的图像信息，帧间预测使用运动信息从参考帧中找到参考块，根据参考块生成预测块，用于消除时间冗余；帧间预测所使用的帧可以为P帧和/或B帧，P帧指的是向前预测帧，B帧指的是双向预测帧。帧间预测使用运动信息从参考帧中找到参考块，根据参考块生成预测块。运动信息包括参考帧所在的参考帧列表，参考帧索引，以及运动矢量。运动矢量可以是整像素的或者是分像素的，如果运动矢量是分像素的，那么需要在参考帧中使用插值滤波做出所需的分像素的块，这里把根据运动矢量找到的参考帧中的整像素或者分像素的块叫参考块。有的技术会直接把参考块作为预测块，有的技术会在参考块的基础上再处理生成预测块。在参考块的基础上再处理生成预测块也可以理解为把参考块作为预测块然后再在预测块的基础上处理生成当前块的预测块。
帧内估计单元212只参考同一帧图像的信息，预测当前码图像块内的像素信息，用于消除空间冗余。帧内预测所使用的帧可以为I帧。
帧内预测有多种预测模式，以国际数字视频编码标准H系列为例，H.264/AVC标准有8种角度预测模式和1种非角度预测模式，H.265/HEVC扩展到33种角度预测模式和2种非角度预测模式。HEVC使用的帧内预测模式有平面模式(Planar)、DC和33种角度模式，共35种预测模式。VVC使用的帧内模式有Planar、DC和65种角度模式，共67种预测模式。
需要说明的是，随着角度模式的增加，帧内预测将会更加精确，也更加符合对高清以及超高清数字视频发展的需求。
残差单元220可基于CU的像素块及CU的PU的预测块来产生CU的残差块。举例来说，残差单元220可产生CU的残差块，使得残差块中的每一采样具有等于以下两者之间的差的值：CU的像素块中的采样，及CU的PU的预测块中的对应采样。
变换/量化单元230可量化变换系数。变换/量化单元230可基于与CU相关联的量化参数(QP)值来量化与CU的TU相关联的变换系数。视频编码器200可通过调整与CU相关联的QP值来调整应用于与CU相关联的变换系数的量化程度。
反变换/量化单元240可分别将逆量化及逆变换应用于量化后的变换系数，以从量化后的变换系数重建残差块。
重建单元250可将重建后的残差块的采样加到预测单元210产生的一个或多个预测块的对应采样，以产生与TU相关联的重建图像块。通过此方式重建CU的每一个TU的采样块，视频编码器200可重建CU的像素块。
环路滤波单元260用于对反变换与反量化后的像素进行处理，弥补失真信息，为后续编码像素提供更好的参考，例如可执行消块滤波操作以减少与CU相关联的像素块的块效应。
在一些实施例中，环路滤波单元260包括去块滤波单元和样点自适应补偿/自适应环路滤波(SAO/ALF)单元，其中去块滤波单元用于去方块效应，SAO/ALF单元用于去除振铃效应。
解码图像缓存270可存储重建后的像素块。帧间预测单元211可使用含有重建后的像素块的参考图像来对其它图像的PU执行帧间预测。另外，帧内估计单元212可使用解码图像缓存270中的重建后的像素块来对在与CU相同的图像中的其它PU执行帧内预测。
熵编码单元280可接收来自变换/量化单元230的量化后的变换系数。熵编码单元280可对量化后的变换系数执行一个或多个熵编码操作以产生熵编码后的数据。
图3是本申请实施例涉及的视频解码器的示意性框图。
如图3所示，视频解码器300包含：熵解码单元310、预测单元320、反量化/变换单元330、重建单元340、环路滤波单元350及解码图像缓存360。需要说明的是，视频解码器300可包含更多、更少或不同的功能组件。
视频解码器300可接收码流。熵解码单元310可解析码流以从码流提取语法元素。作为解析码流的一部分，熵解码单元310可解析码流中的经熵编码后的语法元素。预测单元320、反量化/变换单元330、重建单元340及环路滤波单元350可根据从码流中提取的语法元素来解码视频数据，即产生解码后的视频数据。
在一些实施例中，预测单元320包括帧间预测单元321和帧内估计单元322。
帧内估计单元322可执行帧内预测以产生PU的预测块。帧内估计单元322可使用帧内预测模式以基于空间相邻PU的像素块来产生PU的预测块。帧内估计单元322还可根据从码流解析的一个或多个语法元素来确定PU的帧内预测模式。
帧间预测单元321可根据从码流解析的语法元素来构造第一参考图像列表(列表0)及第二参考图像列表(列表1)。此外，如果PU使用帧间预测编码，则熵解码单元310可解析PU的运动信息。帧间预测单元321可根据PU的运动信息来确定PU的一个或多个参考块。帧间预测单元321可根据PU的一个或多个参考块来产生PU的预测块。
反量化/变换单元330可逆量化(即，解量化)与TU相关联的变换系数。反量化/变换单元330可使用与TU的CU相关联的QP值来确定量化程度。
在逆量化变换系数之后，反量化/变换单元330可将一个或多个逆变换应用于逆量化变换系数，以便产生与TU相关联的残差块。
重建单元340使用与CU的TU相关联的残差块及CU的PU的预测块以重建CU的像素块。例如，重建单元340可将残差块的采样加到预测块的对应采样以重建CU的像素块，得到重建图像块。
环路滤波单元350可执行消块滤波操作以减少与CU相关联的像素块的块效应。
视频解码器300可将CU的重建图像存储于解码图像缓存360中。视频解码器300可将解码图像缓存360中的重建图像作为参考图像用于后续预测，或者，将重建图像传输给显示装置呈现。
视频编解码的基本流程如下：在编码端，将一帧图像划分成块，针对当前块，预测单元210使用帧内预测或帧间预测产生当前块的预测块。残差单元220可基于预测块与当前块的原始块计算残差块，即预测块和当前块的原始块的差值，该残差块也可称为残差信息。该残差块经由变换/量化单元230变换与量化等过程，可以去除人眼不敏感的信息，以消除视觉冗余。可选的，经过变换/量化单元230变换与量化之前的残差块可称为时域残差块，经过变换/量化单元230变换与量化之后的时域残差块可称为频率残差块或频域残差块。熵编码单元280接收到变化量化单元230输出的量化后的变化系数，可对该量化后的变化系数进行熵编码，输出码流。例如，熵编码单元280可根据目标上下文模型以及二进制码流的概率信息消除字符冗余。
在解码端，熵解码单元310可解析码流得到当前块的预测信息、量化系数矩阵等，预测单元320基于预测信息对当前块使用帧内预测或帧间预测产生当前块的预测块。反量化/变换单元330使用从码流得到的量化系数矩阵，对量化系数矩阵进行反量化、反变换得到残差块。重建单元340将预测块和残差块相加得到重建块。重建块组成重建图像，环路滤波单元350基于图像或基于块对重建图像进行环路滤波，得到解码图像。编码端同样需要和解码端类似的操作获得解码图像。该解码图像也可以称为重建图像，重建图像可以为后续的帧作为帧间预测的参考帧。
需要说明的是，编码端确定的块划分信息，以及预测、变换、量化、熵编码、环路滤波等模式信息或者参数信息等在必要时携带在码流中。解码端通过解析码流及根据已有信息进行分析确定与编码端相同的块划分信息，预测、变换、量化、熵编码、环路滤波等模式信息或者参数信息，从而保证编码端获得的解码图像和解码端获得的解码图像相同。
上述是基于块的混合编码框架下的视频编解码器的基本流程，随着技术的发展，该框架或流程的一些模块或步骤可能会被优化，本申请适用于该基于块的混合编码框架下的视频编解码器的基本流程，但不限于该框架及流程。
在一些实施例中，当前块(current block)可以是当前编码单元(CU)或当前预测单元(PU)等。由于并行处理的需要，图像可以被划分成片slice等，同一个图像中的片slice可以并行处理，也就是说它们之间没有数据依赖。而“帧”是一种常用的说法，一般可以理解为一帧是一个图像。在申请中所述帧也可以替换为图像或slice等。
目前正在制定中的多功能视频编码(Versatile Video Coding，VVC)视频编解码标准中，有一个叫做几何划分预测模式(GeometricpartitioningMode，GPM)的帧间预测模式。目前正在制定中的视频编解码标准(Audio Video coding Standard，AVS)视频编解码标准中，有一个叫做角度加权预测模式(Angular Weightedprediction，AWP)的帧间预测模式。这两种模式虽然名称不同、具体的实现形式不同、但原理上有共通之处。
需要说明的是，传统的单向预测只是查找一个与当前块大小相同的参考块，而传统的双向预测使用两个与当前块大小相同的参考块，且预测块内每个点的像素值为两个参考块对应位置的平均值，即每一个参考块的所有点都占50％的比例。双向加权预测使得两个参考块的比例可以不同，如第一个参考块中所有点都占75％的比例，第二个参考块中所有点都占25％的比例。但同一个参考块中的所有点的比例都相同。其他一些优化方式比如采用解码端运动矢量修正(Decoder sideMotion Vector Refinement，DMVR)技术、双向光流(Bi-directional Optical Flow，BIO)等会使参考像素或预测像素产生一些变化，而且GPM或AWP也使用两个与当前块大小相同的参考块，但某些像素位置100％使用第一个参考块对应位置的像素值，某些像素位置100％使用第二个参考块对应位置的像素值，而在交界区域，按一定比例使用这两个参考块对应位置的像素值。具体这些权重如何分配，由GPM或AWP的预测模式决定，或者也可以认为GPM或AWP使用两个与当前块大小不相同的参考块，即各取所需的一部分作为参考块。即将权重不为0的部分作为参考块，而将权重为0的部分剔除出来。
示例性地，图4为权重分配示意图，如图4所示，其示出了本申请实施例提供的一种GPM在64×64的当前块上的多种划分模式的权重分配示意图，其中，GPM存在有64种划分模式。图5为权重分配示意图，如图5所示，其示出了本申请实施例提供的一种AWP在64×64的当前块上的多种划分模式的权重分配示意图，其中，AWP存在有56种划分模式。无论是图4还是图5，每一种划分模式下，黑色区域表示第一个参考块对应位置的权重值为0％，白色区域表示第一个参考块对应位置的权重值为100％，灰色区域则按颜色深浅的不同表示第一个参考块对应位置的权重值为大于0％且小于100％的某一个权重值，第二个参考块对应位置的权重值则为100％减去第一个参考块对应位置的权重值。
GPM和AWP的权重导出方法不同。GPM根据每种模式确定角度及偏移量，而后计算出每个模式的权重矩阵。AWP首先做出一维的权重的线，然后使用类似于帧内角度预测的方法将一维的权重的线铺满整个矩阵。
应理解，早期的编解码技术中只存在矩形的划分方式，无论是CU、PU还是变换单元(Transform Unit，TU)的划分。而GPM或AWP均在没有划分的情况下实现了预测的非矩形的划分效果。GPM和AWP使用了两个参考块的权重的蒙版(mask)，即上述的权重图。这个蒙版确定了两个参考块在产生预测块时的权重，或者可以简单地理解为预测块的一部分位置来自于第一个参考块一部分位置来自于第二个参考块，而过渡区域(blending area)用两个参考块的对应位置加权得到，从而使过渡更平滑。GPM和AWP没有按划分线把当前块划分成两个CU或PU，于是在预测之后的残差的变换、量化、反变换、反量化等也都是将当前块作为一个整体来处理。
GPM使用权重矩阵模拟了几何形状的划分，更确切地说是模拟了预测的划分。而要实施GPM，除了权重矩阵还需要2个预测值，每个预测值由1个单向运动信息确定。这2个单向运动信息来自于一个运动信息候选列表，例如来自merge运动信息候选列表(mergeCandList)。GPM在码流中使用两个索引从mergeCandList中确定2个单向运动信息。
帧间预测使用运动信息(motion information)来表示“运动”。基本的运动信息包含参考帧(reference frame)(或者叫参考图像(reference picture))的信息和运动矢量(MV,motion vector)的信息。常用的双向预测，使用2个参考块对当前块进行预测。2个参考块可以使用一个前向的参考块和一个后向的参考块。可选的，也允许2个都是前向或2个都是后向。所谓前向指参考帧对应的时刻在当前帧之前，后向指参考帧对应的时刻在当前帧之后。或者说前向指参考帧在视频中的位置位于当前帧之前，后向指参考帧在视频中的位置位于当前帧之后。或者说前向指参考帧的POC(picture order count)小于当前帧的POC，后向指参考帧的POC大于当前帧的POC。为了能使用双向预测，自然需要能找到2个参考块，那么就需要2组参考帧的信息和运动矢量的信息。可以把它们每一组理解为一个单向运动信息，而把这2组组合到一起就形成了一个双向运动信息。在具体实现时，单向运动信息和双向运动信息可以使用相同的数据结构，只是双向运动信息的2组参考帧的信息和运动矢量的信息都有效，而单向运动信息的其中一组参考帧的信息和运动矢量的信息是无效的。
在一些实施例中，支持2个参考帧列表，记为RPL0，RPL1，其中RPL是Reference Picture List的简写。在一些实施例中，P slice只可以使用RPL0，B slice可以使用RPL0和RPL1。对一个slice，每个参考帧列表中有若干参考帧，编解码器通过参考帧索引来找到某一个参考帧。在一些实施例中，用参考帧索引和运动矢量来表示运动信息。如对上述的双向运动信息，使用参考帧列表0对应的参考帧索引refIdxL0，以及参考帧列表0对应的运动矢量mvL0，参考帧列表1对应的参考帧索引refIdxL1，以及参考帧列表1对应的运动矢量mvL0。这里的参考帧列表0对应的参考帧索引，参考帧列表1对应的参考帧索引就可以理解为上述的参考帧的信息。在一些实施例中，用两个标志位来分别表示是否使用参考帧列表0对应的运动信息以及是否使用参考帧列表0对应的运动信息，分别记为predFlagL0和predFlagL1。也可以理解为predFlagL0和predFlagL1表示上述单向运动信息“是否有效”。虽然没有明确地提到运动信息这种数据结构，但是它用每个参考帧列表对应的参考帧索引，运动矢量以及“是否有效”的标志位一起来表示运动信息。在一些标准文本中不出现运动信息，而是使用的运动矢量，也可以认为参考帧索引和是否使用对应运动信息的标志是运动矢量的附属。本申请中为了描述方便仍然用“运动信息”，但是应当理解，也可以用“运动矢量”来描述。
当前块所使用的运动信息可以保存下来。当前帧的后续编解码的块可以根据相邻的位置关系使用前面已编解码的块，如相邻块，的运动信息。这利用了空域上的相关性，所以这种已编解码的运动信息叫做空域上的运动信息。当前帧的每个块所使用的运动信息可以保存下来。后续编解码的帧可以根据参考关系使用前面已编解码的帧的运动信息。这利用了时域上的相关性，所以这种已编解码的帧的运动信息叫做时域上的运动信息。当前帧的每个块所使用的运动信息的存储方法通常将一个固定大小的矩阵，如4x4的矩阵，作为一个最小单元，每个最小单元单独存储一组运动信息。这样每编解码一个块，它的位置对应的那些最小单元就可以把这个块的运动信息存储下来。这样使用空域上的运动信息或时域上的运动信息时可以直接根据位置找到该位置对应的运动信息。如一个16x16的块使用了传统的单向预测，那么这个块对应的所有的4x4个最小单元都存储这个单向预测的运动信息。如果一个块使用了GPM或AWP，那么这个块对应的所有的最小单元会根据GPM或AWP的模式，第一个运动信息，和第二个运动信息以及每个最小单元的位置确定每个最小单元存储的运动信息。一种方法是如果一个最小单元对应的4x4的像素全部来自于第一个运动信 息，那么这个最小单元存储第一个运动信息，如果一个最小单元对应的4x4的像素全部来自于第二个运动信息，那么这个最小单元存储第二个运动信息。如果一个最小单元对应的4x4的像素既来自于第一个运动信息又来自于第二个运动信息，那么AWP会选择其中一个运动信息进行存储；GPM的做法是如果两个运动信息指向不同的参考帧列表，那么把它们组合成双向运动信息存储，否则只存储第二个运动信息。
可选的，上述mergeCandList是根据空域运动信息，时域运动信息，基于历史的运动信息，还有一些其他的运动信息来构建的。示例性的，mergeCandList使用如图6A中1至5的位置来推导空域运动信息，使用如图6A中的6或7的位置来推导时域运动信息。基于历史的运动信息是在每编解码一个块时，把这个块的运动信息添加到一个先进先出的列表里，添加过程可能需要一些检查，如是否跟列表里现有的运动信息重复。这样在编解码当前块时就可以参考这个基于历史的列表里的运动信息。
在一些实施例中，关于GPM的语法描述如表1所示：
表1
Figure PCTCN2022086453-appb-000001
如表1所示，在merge模式下，如果regular_merge_flag不为1，当前块可能用CIIP或GPM。如果当前块不用CIIP，那么它就用GPM，也就是在表1中的语法“if(！ciip_flag[x0][y0])”所示的内容。
如上述表1可知，GPM需要在码流中传输3个信息，即merge_gpm_partition_idx，merge_gpm_idx0，merge_gpm_idx1。x0，y0用来确定当前块左上角亮度像素相对于图像左上角亮度像素的坐标(x0,y0)。merge_gpm_partition_idx确定GPM的划分形状，如上所示，它是“模拟划分”，merge_gpm_partition_idx为本申请实施例所说的权重导出模式或者说权重导出模式的索引。merge_gpm_idx0是第一个运动信息在候选列表中的索引值，merge_gpm_idx1是第二个运动信息在候选列表中的索引值。如果候选列表长度(MaxNumGpmMergeCand)>2才需要传merge_gpm_idx1，否则可以直接确定。
在一些实施例中，GPM的解码过程包括如下步骤：
解码过程输入的信息包括：当前块左上角的亮度位置相对于图像左上角的坐标(xCb,yCb)，当前块亮度分量的宽度cbWidth，当前块亮度分量的高度cbHeight，1/16像素精度的亮度运动矢量mvA和mvB，色度运动矢量mvCA和mvCB，参考帧索引refIdxA和refIdxB，预测列表标志predListFlagA和predListFlagB。
示例性的，可以用运动矢量，参考帧索引和预测列表标志组合起来来表示运动信息。在一些实施例中，支持2个参考帧列表，每个参考帧列表可能有多个参考帧。单向预测只使用其中一个参考帧列表中的一个参考帧的一个参考块作为参考，双向预测使用两个参考帧列表中的各一个参考帧的各一个参考块作为参考。而GPM使用2各单向预测。上述mvA和mvB，mvCA和mvCB，refIdxA和refIdxB，predListFlagA和predListFlagB中的A可以理解为第一预测模式，B可以理解为第二预测模式。可选的，用X表示A或B，predListFlagX表示X使用第一个参考帧列表还是第二个参考帧列表，refIdxX表示X使用的参考帧列表中的参考帧索引，mvX表示X使用的亮度运动矢量，mvCX表示X使用的色度运动矢量。需要说明的是，可以认为使用运动矢量，参考帧索引和预测列表标志组合起来来表示本申请所述的运动信息。
解码过程输出的信息包括：(cbWidth)X(cbHeight)的亮度预测样本矩阵predSamplesL；(cbWidth/SubWidthC)X(cbHeight/SubHeightC)的Cb色度分量的预测样本矩阵，如果需要；(cbWidth/SubWidthC)X(cbHeight/SubHeightC) 的Cr色度分量的预测样本矩阵，如果需要。
示例性的，下面以亮度分量举例，色度分量的处理和亮度分量类似。
假设predSamplesLAL和predSamplesLBL的大小为(cbWidth)X(cbHeight)，为根据2个预测模式做出的预测样本矩阵。predSamplesL按如下方法导出：分别根据亮度运动矢量mvA和mvB，色度运动矢量mvCA和mvCB，参考帧索引refIdxA和refIdxB，预测列表标志predListFlagA和predListFlagB确定predSamplesLAL和predSamplesLBL。即分别根据2个预测模式的运动信息进行预测，详细过程不再赘述。通常GPM是merge模式，可以认为GPM的2个预测模式都是merge模式。
根据merge_gpm_partition_idx[xCb][yCb]，利用表2确定GPM的“划分”角度索引变量angleIdx和距离索引变量distanceIdx。
表2–angleIdx和distanceIdx与merge_gpm_partition_idx的对应关系
merge_gpm_partition_idx 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
angleIdx 0 0 2 2 2 2 3 3 3 3 4 4 4 4 5 5
distanceIdx 1 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1
merge_gpm_partition_idx 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
angleIdx 5 5 8 8 11 11 11 11 12 12 12 12 13 13 13 13
distanceIdx 2 3 1 3 0 1 2 3 0 1 2 3 0 1 2 3
merge_gpm_partition_idx 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
angleIdx 14 14 14 14 16 16 18 18 18 19 19 19 20 20 20 21
distanceIdx 0 1 2 3 1 3 1 2 3 1 2 3 1 2 3 1
merge_gpm_partition_idx 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
angleIdx 21 21 24 24 27 27 27 28 28 28 29 29 29 30 30 30
distanceIdx 2 3 1 3 1 2 3 1 2 3 1 2 3 1 2 3
需要说明的是，因为三个分量(component，如Y、Cb、Cr)都可以使用GPM，所以在一些标准文本将一个分量产生GPM的预测样本矩阵的过程分装到了一个子流程里面，即GPM的加权预测过程(Weighted sample prediction process for geometric partitioning mode)，三个分量都会调用这个流程，只是调用的参数不同，这里只用亮度分量举例。当前亮度块的预测矩阵predSamplesL[xL][yL](其中xL＝0..cbWidth–1，yL＝0..cbHeight-1)由GPM的加权预测过程导出。其中nCbW设为cbWidth,nCbH设为cbHeight，两个预测模式做的预测样本矩阵predSamplesLAL和predSamplesLBL,还有angleIdx,distanceIdx作为输入。
在一些实施例中，GPM的加权预测导出过程包括如下步骤：
该过程的输入有：当前块的宽度nCbW，当前块的高度nCbH；2个(nCbW)X(nCbH)的预测样本矩阵predSamplesLA和predSamplesLB；GPM的“划分”角度索引变量angleIdx；GPM的距离索引变量distanceIdx；分量索引变量cIdx。本示例以亮度举例，因此上述cIdx为0，表示亮度分量。
该过程的输出有：(nCbW)X(nCbH)的GPM预测样本矩阵pbSamples。
示例性的，变量nW,nH,shift1,offset1,displacementX,displacementY,partFlip还有shiftHor按如下方法导出:
nW＝(cIdx＝＝0)？nCbW:nCbW*SubWidthC；
nH＝(cIdx＝＝0)？nCbH:nCbH*SubHeightC；
shift1＝Max(5,17-BitDepth)，其中BitDepth是编解码的比特深度；
offset1＝1<<(shift1-1)，其中“<<”表示左移；
displacementX＝angleIdx；
displacementY＝(angleIdx+8)％32；
partFlip＝(angleIdx>＝13&&angleIdx<＝27)？0:1；
shiftHor＝(angleIdx％16＝＝8||(angleIdx％16！＝0&&nH>＝nW))？0:1。
变量offsetX和offsetY按如下方法导出:
当shiftHor的值为0时:
offsetX＝(-nW)>>1，
offsetY＝((-nH)>>1)+(angleIdx<16？(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3))。
当shiftHor的值为1时:
offsetX＝((-nW)>>1)+(angleIdx<16？(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)，
offsetY＝(-nH)>>1。
变量xL和yL按如下方法导出:
xL＝(cIdx＝＝0)？x:x*SubWidthC，
yL＝(cIdx＝＝0)？y:y*SubHeightC，
表示当前位置预测样本权重的变量wValue按如下方法导出，wValue即为(x，y)点的第一预测模式的预测矩阵的预测值predSamplesLA[x][y]的权重，而(8-wValue)即为(x，y)点的第一预测模式的预测矩阵的预测值predSamplesLB[x][y]的权重。
其中距离矩阵disLut按表3确定:
表3
idx 0 2 3 4 5 6 8 10 11 12 13 14
disLut[idx] 8 8 8 4 4 2 0 -2 -4 -4 -8 -8
idx 16 18 19 20 21 22 24 26 27 28 29 30
disLut[idx] -8 -8 -8 -4 -4 -2 0 2 4 4 8 8
weightIdx＝(((xL+offsetX)<<1)+1)*disLut[displacementX]+(((yL+offsetY)<<1)+1)*disLut[displacementY]，
weightIdxL＝partFlip？32+weightIdx:32–weightIdx，
wValue＝Clip3(0,8,(weightIdxL+4)>>3)，
预测样本的值pbSamples[x][y]按如下方法导出:
pbSamples[x][y]＝Clip3(0,(1<<BitDepth)-1,(predSamplesLA[x][y]*wValue+predSamplesLB[x][y]*(8-wValue)+offset1)>>shift1)。
需要说明的是，对当前块的每一个位置推导一个权重值，然后计算一个GPM的预测值pbSamples[x][y]。因为这种方式权重wValue不必写成一个矩阵的形式，但是可以理解如果把每个位置的wValue都保存到一个矩阵里，那它就是一个权重矩阵。每个点分别计算权重并加权得到GPM的预测值，或者计算出所有的权重再统一加权得到GPM的预测样本矩阵其原理是一样的。而本申请中的诸多描述中使用权重矩阵的说法，是为了表述更容易理解，用权重矩阵画图更直观，其实也可以按每个位置的权重来描述。比如权重矩阵导出模式也可以说成权重导出模式。
在一些实施例中，如图6B所示，GPM的解码流程可以表述为：解析码流，确定当前块是否使用GPM技术；如果当前块使用GPM技术，确定权重导出模式(或“划分”模式或权重矩阵导出模式)，及第一运动信息和第二运动信息。分别根据第一运动信息确定第一预测块，根据第二运动信息确定第二预测块，根据权重矩阵导出模式确定权重矩阵，根据第一预测块和第二预测块和权重矩阵确定当前块的预测块。
需要说明的是，在本申请的实施例中，GPM或AWP属于一种预测技术，GPM或AWP需要在码流中传输一个GPM或AWP是否使用的标志(flag)，该flag可以指示当前块是否使用GPM或AWP。如果使用GPM或AWP，编码器在码流中需要传输具体使用的模式，即GPM的64种划分模式之一，或AWP的56种划分模式之一；以及两个单向运动信息的索引值。也就是说，对于当前块而言，解码器通过解析码流可以得到GPM或AWP是否使用的信息，如果确定使用GPM或AWP，解码器可以解析出GPM或AWP的预测模式参数以及两个运动信息索引值，比如当前块可以划分为两个分区，那么可以解析出第一分区对应的第一索引值和第二分区对应的第二索引值。
具体来讲，对于GPM模式来说，如果使用GPM，那么码流中将会传输GPM下的预测模式参数，比如GPM具体的划分模式；通常情况下，GPM包括有64种划分模式。对于AWP模式来说，如果使用AWP，那么码流中将会传输AWP下的预测模式参数，比如AWP具体的划分模式；通常情况下，AWP包括有56种划分模式。
在帧间预测模式下，比如GPM和AWP均需要使用两个单向运动信息查找两个参考块。目前的实现方式是在编码器侧利用当前块之前已编码/已解码部分的相关信息构建一个单向运动信息候选列表，从单向运动信息候选列表中选择单向运动信息，将这两个单向运动信息在单向运动信息候选列表中的索引值(index)写入码流。在解码器侧采用同样的方式，即利用当前块之前已解码部分的相关信息构建一个单向运动信息候选列表，这个单向运动信息候选列表与编码器侧构建的候选列表一定是相同的。如此，从码流中解析出两个单向运动信息的索引值，然后从单向运动信息候选列表中查找出这两个单向运动信息即为当前块需要使用的两个单向运动信息。
也就是说，本申请所描述的单向运动信息可以包括：运动矢量信息，即(x，y)的值，以及对应的参考帧信息，即参考帧列表及在参考帧列表中的参考帧索引值。一种表示方式是记录两个参考帧列表的参考帧索引值，其中一个参考帧列表对应的参考帧索引值有效，如0，1，2等；另一个参考帧列表对应的参考帧索引值为无效，即-1。参考帧索引值有效的参考帧列表即为当前块的运动信息所使用的参考帧列表，根据参考帧索引值可以从该参考帧列表中查找到对应的参考帧。每个参考帧列表都有一个对应的运动矢量，有效的参考帧列表对应的运动矢量是有效的，无效的参考帧列表对应的运动矢量是无效的。解码器可以通过单向运动信息中的参考帧信息找到所需的参考帧，根据当前块的位置以及运动矢量即(x，y)的值可以在参考帧中找到参考块，进而确定出当前块的帧间预测值。
帧内预测方法是使用当前块周边已编解码的重建像素作为参考像素来对当前块进行预测。图7A为帧内预测的示意图，如图7A所示，当前块的大小为4x4，当前块左边一行和上面一列的像素为当前块的参考像素，帧内预测使用这些参考像素对当前块进行预测。这些参考像素可能已经全部可得，即全部已经编解码。也可能有部分不可得，比如当前块是整帧的最左侧，那么当前块的左边的参考像素不可得。或者编解码当前块时，当前块左下方的部分还没有编解码，那么左下方的参考像素也不可得。对于参考像素不可得的情况，可以使用可得的参考像素或某些值或某些方法进行填充，或者不进行填充。
图7B为帧内预测的示意图，如图7B所示，多参考行帧内预测方法(Multiple reference line，MRL)可以使用更多的参考像素从而提高编解码效率，例如，使用4个参考行/列为当前块的参考像素。
进一步地，帧内预测有多种预测模式，图8A-5I为帧内预测的示意图，如图8A-5I所示，H.264中对4x4的块进行帧内预测主要可以包括9种模式。其中，如图8A所示的模式0将当前块上面的像素按垂直方向复制到当前块作为预测值，如图8B所示的模式1将左边的参考像素按水平方向复制到当前块作为预测值，如图8C所示的模式2直流DC将A～D和I～L这8个点的平均值作为所有点的预测值，如图8D-5I所示的模式3～8分别按某一个角度将参考像素复制到当前块的对应位置，因为当前块某些位置不能正好对应到参考像素，可能需要使用参考像素的加权平均值，或者说是插值的参考像素的分像素。
除此之外，还有Planar模式等，而随着技术的发展以及块的扩大，角度预测模式也越来越多。图9为帧内预测模式的示意图，如图9所示，如HEVC使用的帧内预测模式有Planar、DC和33种角度模式共35种预测模式。图10为帧内预测模式的示意图，如图10所示，VVC使用的帧内模式有Planar、DC和65种角度模式共67种预测模式。图11为帧内预测模式的示意图，如图11所示，AVS3使用DC、Planar、Bilinear和63种角度模式共66种预测模式。
另外还有一些技术对预测进行改进，如改进参考像素的分像素插值，对预测像素进行滤波等。如AVS3中的多组合帧内预测滤波(multipleintraprediction filter，MIPF)对不同的块大小，使用不同的滤波器产生预测值。对同一个块内的不同位置的像素，与参考像素较近的像素使用一种滤波器产生预测值，与参考像素较远的像素使用另一种滤波器产生预测值。对预测像素进行滤波的技术如AVS3中的帧内预测滤波(intraprediction filter，IPF)，对预测值可以使用参考像素进行滤波。
在帧内预测中可以使用最可能模式列表(MostprobableModes List，MPM)的帧内模式编码技术来提高编解码效率。利用周边已编解码的块的帧内预测模式，以及根据周边已编解码块的帧内预测模式导出的帧内预测模式，如相邻的模式，以及一些常用或使用概率比较高的帧内预测模式，如DC，Planar，Bilinear模式等，构成一个模式列表。参考周边已编解码的块的帧内预测模式利用了空间上的相关性。因为纹理在空间上会有一定的连续性。MPM可以作为帧内预测模式的预测。也就是认为当前块使用MPM的概率会比不使用MPM的概率高。因而在二值化时，会给MPM使用更少的码字，从而节省开销，以提高编解码效率。
GPM用权重矩阵组合两个帧间预测块。实际上它可以扩展到组合两个任意的预测块。如两个帧间预测块，两个帧内预测块，一个帧间预测块和一个帧内预测块。甚至在屏幕内容编码中，还可以使用IBC(intra block copy)或palette的预测块作为其中的1个或2个预测块。
本申请将帧内、帧间、IBC、palette称为不同的预测方式。为了表述方便，这里使用一个叫预测模式的称呼。预测模式可以理解为根据它编解码器可以产生当前块的一个预测块的信息。比如，在帧内预测中，预测模式可以是某个帧内预测模式，如DC，Planar，各种帧内角度预测模式等。当然也可以叠加某个或某些辅助的信息，比如帧内参考像素的优化方法，产生初步的预测块以后的优化方法(比如滤波)等。比如，在帧间预测中，预测模式可以是skip(跳过)模式，merge(合并)模式或MMVD(merge with motion vector difference，带运动矢量差的合并)模式，或AMVP(advanced motion vector predition，高级运动矢量预)，可以是单向预测也可以是双向预测或多假设预测。如果帧间的预测模式使用单向预测，它要能确定一个运动信息，这个运动信息是一个单向运动信息，根据运动信息能确定预测块。如果帧间的预测模式使用双向预测，它要能确定一个双向运动信息或两个单向运动信息，根据运动信息能确定预测块。如果帧间的预测模式使用多假设预测，它要能确定多个单向运动信息，根据运动信息能确定预测块。skip，merge，普通的inter模式都可以支持单向预测，双向预测或多假设预测。一个预测模式如果是帧间预测模式，它能确定运动信息，根据运动信息能确定预测块。在skip模式和merge模式，MMVD模式，普通inter模式的基础上都可以使用模板匹配的方法，这样的预测模式可以仍然称为skip模式和merge模式，MMVD模式，普通inter模式或者使用模板匹配的skip模式，使用模板匹配的merge模式，使用模板匹配的MMVD模式，使用模板匹配的普通inter模式。
skip模式和merge模式都不需要在码流中传输运动矢量差MVD，skip模式还不需要在码流种传输残差。而MMVD可以认为是一种特殊的merge模式，它通过一些标志位来表示一些特定的MVD，这些特定的MVD只有几种可能的预设值。一个例子是VVC中的MMVD模式，它用mmvd_direction_idx表示MVD的方向，mmvd_direction_idx可能的值是0，1，2，3。0表示MMVD的水平分量为正值，竖直方向为0；1表示MMVD的水平分量为负值，竖直方向为0；2表示MMVD的水平分量为0，竖直方向为正值；3表示MMVD的水平分量为0，竖直方向为负值。用mmvd_distance_idx表示上述正值或负值的绝对值，mmvd_distance_idx可能的值是0～7，在ph_mmvd_fullpel_only_flag＝＝0时分别表示1，2，4，8，16，32，64，128，在ph_mmvd_fullpel_only_flag＝＝1时分别表示4，8，16，32，64，128，256，512。而普通的inter模式的MVD理论上可以表示一个有效范围内的任意可能的MVD。
这样GPM需要确定的信息可以表述为1个权重导出模式和2个预测模式。权重导出模式用来确定权重矩阵或权重，2个预测模式分别确定一个预测块或预测值。权重导出模式在某些地方也被称为划分模式。但因为它是模拟划分，本申请更习惯称为权重导出模式。
可选的，2个预测模式可以来自相同的或不同的预测方式，其中预测方式包括但不限于帧内预测、帧间预测、IBC、palette。
一个具体的具体的例子如下：如果当前块使用GPM。这个例子用在帧间编码的块中，允许使用帧内预测和帧间预测中的merge模式。如表4所示，增加一个语法元素intra_mode_idx表示哪一个预测模式是帧内预测模式，比如intra_mode_idx为0表示2个预测模式都是帧间预测模式，即mode0IsInter为1，mode0IsInter为1；intra_mode_idx为1表示第一预测模式是帧内预测模式，第二预测模式是帧间预测模式，即mode0IsInter为0，mode0IsInter为1；intra_mode_idx为2表示第一预测模式是帧间预测模式，第二预测模式是帧内预测模式，即mode0IsInter为1，mode0IsInter为0；intra_mode_idx为3表示两个预测模式都是帧内预测模式，即mode0IsInter为0，mode0IsInter为0。
表4
Figure PCTCN2022086453-appb-000002
在一些实施例中，如图12所示，GPM的解码流程可以表述为：解析码流，确定当前块是否使用GPM技术；如果当前块使用GPM技术，确定权重导出模式(或“划分”模式或权重矩阵导出模式)，及第一帧内预测模式和第二帧内预测模式。分别根据第一帧内预测模式确定第一预测块，根据第二帧内预测模式确定第二预测块，根据权重矩阵导出模式确定权重矩阵，根据第一预测块和第二预测块和权重矩阵确定当前块的预测块。
模板匹配(template matching)的方法最早用在帧间预测中，它利用相邻像素之间的相关性，把当前块周边的一些区域作为模板。在当前块进行编解码时，按照编码顺序其左侧及上侧已经编解码完成。当然在现有的硬件解码器实现时，不一定能保证当前块开始解码时，其左侧和上侧已经解码完成，当然这里说的是帧间块，比如在HEVC中帧间编码的块产生预测块时是不需要周边的重建像素的，因而帧间块的预测过程可以并行进行。但是帧内编码的块是一定需要左侧和上侧的重建像素作为参考像素的。理论上左侧和上侧是可得的，也就是说硬件设计做相应的调整是可以实现的。相对来说右侧和下侧在现在标准如VVC的编码顺序下是不可得的。
如图13所示把当前块的左侧和上侧的矩形区域设为模板，左侧的模板部分的高度一般和当前块的高度相同，上侧的模板的部分的宽度一般和当前块的宽度相同，当然也可以不同。在参考帧中寻找模板的最佳匹配位置从而确定当前块的运动信息或者说运动矢量。这个过程大致可以描述为，在某一个参考帧中，从一个起始位置开始，在周边一定范围内进行搜索。可以预先设定好搜索的规则，如搜索范围搜索步长等。每移动一个到位置，计算该位置对应的模板和当前块周边的模板的匹配程度，所谓匹配程度可以用一些失真代价来衡量，比如说SAD(sum of absolute difference)，SATD(sum of absolute transformed difference)，一般SATD使用的变换是Hadamard变换，MSE(mean-square error)等，SAD，SATD，MSE等的值越小代表匹配程度越高。用该位置对应的模板的预测块和当前块周边的模板的重建块计算代价。除了整像素位置的搜索还可以进行分像素位置的搜索，根据搜索到的匹配程度最高的位置来确定当前块的运动信息。利用相邻像素之间的相关性，对模板合适的运动信息可能也是当前块合适的运动信息。当然模板匹配的方法可能并不一定对所有的块都适用，因而可以使用一些方法确定当前块是否使用上述模板匹配的方法，比如在当前块用一个控制开关表示是否使用模板匹配的方法。这种模板匹配的方法的一个名字叫DMVD(decoder side motion vector derivation)。编码器和解码器都可以利用模板进行搜索从而导出运动信息或者在原有的运动信息的基础上找到更好的运动信息。而它不需要传输具体的运动矢量或运动矢量差，而是由编码器和解码器都进行同样规则的搜索从而保证编码和解码的一致。模板匹配的方法可以提高压缩性能，但是它需要在解码器中也进行“搜索”，从而带来了一定的解码器复杂度。
上述是在帧间上应用模板匹配的方法，模板匹配的方法也可以用在帧内上，比如说利用模板来确定帧内预测模式。对当前块，同样可以使用当前块上边和左边一定范围内的区域作为模板，比如说仍然如上图所示的左边的矩形区域和上边的矩形区域。在编解码当前块时，在模板中的重建像素是可得的。这个过程大致可以描述为，对当前块确定候选的帧内预测模式的集合，候选的帧内预测模式构成全部可用的帧内预测模式的一个子集。当然候选的帧内预测模式可以是全部可用的帧内预测模式的全集。这可以根据性能和复杂度的权衡来确定。可以根据MPM或一些规则，如等间距筛选等，来确定候选的帧内预测模式的集合。计算各候选的帧内预测模式在模板上的代价，比如SAD，SATD，MSE等。用该模式在模板上进行预测做出预测块，用预测块和模板的重建块计算代价。代价小的模式可能与模板更匹配，利用相邻像素之间的相似性，在模板上表现好的帧内预测模式可能也是当前块上表现好的帧内预测模式。选定1个或几个代价小的模式。当然上述2步可以重复进行，比如说在选定1个或几个代价小的模式后，再一次确定候选的帧内预测模式的集合，对新确定的候选的帧内预测模式集合再计算代价，选定1个或几个代价小的模式。这也可以理解为粗选和细选。最终选定的1个帧内预测模式确定为当前块的帧内预测模式，或者最终选定的几个帧内预测模式作为当前块的帧内预测模式的候选。当然也可以仅仅用模板匹配的方法对候选的帧内预测模式集合进行排序，比如对MPM列表进行排序，即将MPM列表中的模式分别在模板上做出预测块并确定代价，按代价从小到大进行排序。一般MPM列表中越靠前的模式在码流中的开销越小，这样也可以达到提高压缩效率的目的。
模板匹配的方法可以用于确定GPM的2个预测模式上。如果将模板匹配的方法用于GPM，对当前块可以用1个控制开关控制当前块的2个预测模式是否使用模板匹配，也可以用2个控制开关分别控制2个预测模式各自是否使用模板匹配。
另一方面是如何使用模板匹配。比如如果GPM在merge模式下使用，如VVC中的GPM，它使用merge_gpm_idxX从mergeCandList中确定一个运动信息，其中大写的X为0或1。对第X个运动信息，一种方法是在上述运动信息的基础上用模板匹配的方法进行优化。即根据merge_gpm_idxX从mergeCandList中确定一个运动信息，如果对该运动信息使用模板匹配，那么用模板匹配的方法在上述运动信息的基础上进行优化。另一种方法是不使用merge_gpm_idxX从mergeCandList中确定一个运动信息，而是直接从一个默认运动信息的基础上进行搜索，确定一个运动信息。
如果第X预测模式是帧内预测模式，而且当前块的第X预测模式使用模板匹配的方法，那么可以利用模板匹配方法确定一个帧内预测模式，不需要在码流中指示该帧内预测模式的索引。或者利用模板匹配方法确定一个候选集合或者MPM列表，需要在码流中指示该帧内预测模式的索引。
GPM确定了权重导出模式后，就可以确定每个预测模式占领的区域。所谓占领的区域可以理解为该预测模式对应的权重为最大值的区域，或者权重大于或等于某一阈值的区域。GPM之所以能提高压缩性能，就是因为GPM“划分”的2个部分是不同的。所以当用模板匹配的方法来确定GPM的预测模式时，也可以对模板进行划分。现有技术可以将模板分为3类，即左侧，上侧和全部(左侧加上侧)。模板的划分是跟权重导出模式有关的。示例性的，如表5所示，现有技术里模板的划分跟“划分”角度或者说“划分”角度索引angleIdx有关。
表5
“划分”角度索引 第一预测模式对应的模板 第二预测模式对应的模板
0 TM_A TM_AL
1 / /
2 TM_A TM_AL
3 TM_A TM_AL
4 TM_A TM_L
5 TM_AL TM_L
6 / /
7 / /
8 TM_AL TM_L
9 / /
10 / /
11 TM_AL TM_L
12 TM_AL TM_AL
13 TM_A TM_AL
14 TM_A TM_AL
15 / /
16 TM_A TM_AL
17 / /
18 TM_A TM_AL
19 TM_A TM_AL
20 TM_A TM_L
21 TM_AL TM_L
22 / /
23 / /
24 TM_AL TM_L
25 / /
26 / /
27 TM_AL TM_L
28 TM_AL TM_AL
29 TM_A TM_AL
30 TM_A TM_AL
31 / /
例如，记左侧模板为TM_A，上侧模板为TM_L，全部(左侧加上侧)模板为TM_AL。模板和“划分”角度索引的关系如表5所示，其中某些角度索引如1，6，7等在目前的GPM中并没有用到，所以没有对应的模板，用/表示。
下面结合图14，以解码端为例，对本申请实施例提供的视频解码方法进行介绍。
图14为本申请一实施例提供的预测方法流程示意图，本申请实施例应用于图1和图3所示视频解码器。如图14所示，本申请实施例的方法包括：
S101、解码码流，确定K个初始预测模式。
由上述可知，为了进一步提高预测准确性，可以使用多个预测模式对当前块进行预测，此时，为了保持编码端和解码端的预测一致性，在一些实施例中，编码端需要在码流中需要指示权重导出模式，例如在码流中写入权重导出模式的索引，以VVC为例，GPM的64种权重导出模式的编码是按等概率处理的，即64种权重导出模式在进行二值化和反二值化的时候都使用等长的6比特，这样则增加了编码代价。
本申请实施例，为了降低编码代价，解码端通过预测模式导出权重导出模式。具体是，解码端通过解码码流，确定K个初始预测模式，接着，根据这K个初始预测模式，确定当前块的权重导出模式，进而根据当前块的权重导出模式，确定当前块的预测值。
上述当前块的K个初始预测模式包括如下几种示例：
示例1，上述K个初始预测模式均为帧内预测模式。
示例2，上述K个初始预测模式均为帧间预测模式。
示例3，上述K个初始预测模式中，至少一个为帧内预测模式，至少一个为帧间预测模式。
示例4，上述K个初始预测模式中，至少一个为帧内预测模式，至少一个为非帧间和非帧内的预测模式，例如为块内复制IBC预测模式或调色板palette预测模式等。
示例5，上述K个初始预测模式中，至少一个为帧间预测模式，至少一个为非帧间和非帧内的预测模式，例如为IBC预测模式或palette预测模式等。
示例6，上述K个初始预测模式均不是帧内预测模式，也不是帧间预测模式，例如一个为IBC预测模式，一个为palette预测模式等。
需要说明的是，本申请实施例对上述K个初始预测模式的具体类型不做限制。
图15为使用两种预测模式对当前块进行预测时的示意图，如图15所示，在对当前块进行预测时，可以使用第一 预测模式确定第一预测值，同时使用第二预测模式确定第二预测值，然后可以利用权重对第一预测值和第二预测值进行加权处理，得到当前块的预测值。
其中，上述S101中解码端解码码流，确定K个初始预测模式的方式包括但不限于如下几种：
方式1，码流中包括K个初始预测模式的索引，这样解码端通过解码码流，可以得到这K个初始预测模式的索引，进而根据该索引，确定出K个初始预测模式。例如，以K个初始预测模式为帧内预测模式为例，编码端将这K个初始帧内预测模式的索引编入码流中，示例的K＝2，这2个初始预测模式假设为图10所示的VVC的65种角度预测模式中的角度预测模式31和35，这样编码端将索引31和35编入码流。解码端解码码流，得到索引31和35，进而从图10所示的VVC的65种角度预测模式中得到索引31和35对应的帧内预测模式，作为2个初始预测模式。
方式2，上述上述S101包括S101-A1至S101-A3的步骤：
S101-A1、确定备选预测模式列表，该备选预测模式列表中包括至少两个备选预测模式；
S101-A2、解码码流，得到预测模式索引；
S101-A3、根据预测模式索引，从备选预测模式列表中，确定出K个初始预测模式。
在一些实施例中，若上述K个初始预测模式均为帧内预测模式，则该备选预测模式列表所包括的备选预测模式均为帧内预测模式，例如包括直流(Direct Current，DC)模式、平面(PLANAR)模式、角度模式等中的至少一个。可选的，上述备选预测模式列表还可以包括MPM列表中的帧内预测模式。
在一些实施例中，若上述K个初始预测模式均为帧间预测模式，则该备选预测模式列表所包括的备选预测模式均为帧间预测模式，例如包括skip、merge、普通帧间预测模式等中的至少一个。
在一些实施例中，若上述K个初始预测模式中即包括帧间预测模式又包括帧内预测模式，则该备选预测模式列表包括至少一个帧内预测模式和至少一个帧间预测模式。
可选的，上述备选预测模式列表还可以包括IBC、palette等模式。
可选的，上述备选预测模式列表还可以包括单向预测、双向预测、多假设预测等。
本申请对上述备选预测模式列表所包括的备选预测模式的类型和个数不做限制。
本申请实施例对确定备选预测模式列表的方式不做限制。
在一种示例中，上述备选预测模式列表所包括的备选预测模式为预设模式。
在一种示例中，上述备选预测模式列表为MPM列表。
在一种示例中，上述备选预测模式列表是根据一些规则，如等间距筛选等，确定出的预测模式的集合。
在一种示例中，编码端和解码端均默认上述备选预测模式列表中的备选预测模式是通过模板匹配方式确定的，例如，解码端使用预测模式对当前块的模板进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该预测模式的代价，例如SAD代价、SATD代价或MSE代价等。这样解码端可以根据各预测模式的代价对预测模式进行排序，得到备选预测模式列表，例如将代价最小的多个预测模式确定为备选预测模式，构成备选预测模式列表。
本申请实施例中，码流中所包括的预测模式索引可以是K个初始预测模式中至少一个初始预测模式的索引，例如该预测模式索引为K个初始预测模式的索引，或者K个初始预测模式中第一个初始预测模式的索引。解码端解码码流，得到该预测模式索引，并根据该预测模式索引从上述确定的备选预测模式列表中，确定出K个初始预测模式。
示例性的，若码流中所包括的预测模式索引为K个初始预测模式的索引，则解码端将备选预测模式列表中该K个初始预测模式的索引对应的K个备选预测模式，确定为K个初始预测模式。
示例性的，若码流中所包括的预测模式索引为K个初始预测模式中第一个初始预测模式的索引，则解码端将备选预测模式列表中该索引对应的备选预测模式确定为K个初始预测模式中的第一个初始预测模式，将备选预测模式列表中该索引之后的K-1个的索引对应的K-1个备选预测模式，确定为K个初始预测模式中剩余的K-1个初始预测模式。
本申请实施例中，解码端根据上述步骤，确定出K个初始预测模式，根据这K个初始预测模式确定当前块的权重导出模式，再根据该当前块的权重导出模式对当前块进行预测，确定当前块的预测值。
在一些实施例中，解码端在确定K个初始预测模式之前，首先需要判断当前块是否使用K种不同的预测模式进行加权预测处理。若解码端确定当前块使用K种不同的预测模式进行加权预测处理时，则执行上述S101确定K个初始预测模式。
在一种可能的实现方式中，解码端可以通过确定当前块的预测模式参数，来确定当前块是否使用两种不同的预测模式进行加权预测处理。
可选的，在本申请的实施中，预测模式参数可以指示当前块是否可以使用GPM模式或AWP模式，即指示当前块是否可以使用K种不同的预测模式进行预测处理。
可以理解的是，在本申请的实施例中，可以将预测模式参数理解为一个表明是否使用了GPM模式或AWP模式标志位。具体地，编码端可以使用一个变量作为预测模式参数，从而可以通过对该变量的取值的设置来实现预测模式参数的设置。示例性的，在本申请中，如果当前块使用GPM模式或AWP模式，那么编码端可以将预测模式参数的取值设置为指示当前块使用GPM模式或AWP模式，具体地，编码端可以将变量的取值设置为1。示例性的，在本申请中，如果当前块不使用GPM模式或AWP模式，那么编码端可以将预测模式参数的取值设置为指示当前块不使用GPM模式或AWP模式，具体地，编码端可以将变量取值设置为0。进一步地，在本申请的实施例中，编码端在完成对预测模式参数的设置之后，便可以将预测模式参数写入码流中，传输至解码端，从而可以使解码端在解析码流之后获得预测模式参数。
基于此，解码端解码码流，得到预测模式参数，进而根据该预测模式参数确定当前块是否使用GPM模式或AWP模式，若当前块使用GPM模式或AWP模式，即使用K个不同的预测模式进行预测处理时，确定K个初始预测模式。
需要说明的是，在本申请的实施例中，GPM模式或AWP模式为一种预测方法，具体地，为当前块确定K个不同的预测模式，然后分别根据这K个不同的预测模式确定出K个预测值，接着可以再确定权重，将则会K个预测值依据权重进行组合，最终便可以得到当前块的预测值。
在一些实施例中，在应用GPM模式或AWP模式时，可以对当前块的尺寸进行限制。可以理解的是，由于本申请实施例提出的预测方法需要分别使用K个不同的预测模式生成K个预测值，再根据权重进行加权得到当前块的预测值，为了降低的复杂度，同时考虑压缩性能和复杂度的权衡，在本申请的实施例中，可以限制对一些大小的块不使用该GPM模式或AWP模式。因此，在本申请中，解码端可以先确定当前块的尺寸参数，然后根据尺寸参数确定当前块是否使用GPM模式或AWP模式。
需要说明的是，在本申请的实施例中，当前块的尺寸参数可以包括当前块的高度和宽度，因此，解码端可以利用当前块的高度和宽度对使用GPM模式或AWP模式进行限制。示例性的，在本申请中，若宽度大于第一阈值且高度大于第二阈值，则确定当前块使用GPM模式或AWP模式。可见，一种可能的限制是仅仅在块的宽度大于(或大于等于)第一阈值，且块的高度大于(或大于等于)第二阈值的情况下使用GPM模式或AWP模式。其中，第一阈值和第二阈值的值可以是8，16，32等，第一阈值可以等于第二阈值。示例性的，在本申请中，若宽度小于第三阈值且高度大于第四阈值，则确定当前块使用GPM模式或AWP模式。可见，一种可能的限制是仅仅在块的宽度小于(或小于等于)第三阈值，且块的高度大于(或大于等于)第四阈值的情况下使用AWP模式。其中，第三阈值和第四阈值的值可以是8，16，32等，第三阈值可以等于第四阈值。
在一些实施例中，在本申请的实施例中，还可以通过像素参数的限制来实现限制能够使用GPM模式或AWP模式的块的尺寸。示例性的，在本申请中，解码端可以先确定当前块的像素参数，然后再根据像素参数和第五阈值进一步判断当前块是否可以使用GPM模式或AWP模式。可见，一种可能的限制是仅仅在块的像素数大于(或大于等于)第五阈值的情况下使用GPM模式或AWP模式。其中，第五阈值的值可以是8，16，32等。也就是说，在本申请中，只有在当前块的尺寸参数满足大小要求的条件下，当前块才可以使用GPM模式或AWP模式。
在一些实施例中，在本申请中，可以有一个帧级的标志来确定当前待解码帧是否使用本申请的技术方案。如可以配置帧内帧(如I帧)使用本申请，帧间帧(如B帧、P帧)不使用本申请。或者可以配置帧内帧不使用本申请，帧间帧使用本申请。或者可以配置某些帧间帧使用本申请，某些帧间帧不使用本申请。帧间帧也可以使用帧内预测，因而帧间帧也有可能使用本申请。
在一些实施例中，还可以有一个帧级以下、CU级以上(如tile、slice、patch、LCU等)的标志来确定这一区域是否使用本申请的技术方案。
S102、根据K个初始预测模式，确定当前块的权重导出模式。
在本申请中，权重导出模式用于对当前块使用的权重进行确定。具体地，权重导出模式可以是导出权重的模式。对于一个给定长度和宽度的块，每一种权重导出模式可以导出一个权重矩阵；对于同样大小的块，不同权重导出模式导出的权重矩阵不同。
示例性的，在本申请中，AWP有56种权重导出模式，GPM有64种权重导出模式。
本申请实施例中，解码端根据上述确定的K个初始预测模式，确定当前块的权重导出模式，在一些情况下，编码端可以在码流中不写入当前块的权重导出模式相关的信息，进而节省码字，降低编码代价。
本申请实施例对解码端根据K个初始预测模式，确定当前块的权重导出模式的方式不做限制。
在一些实施例中，解码端基于当前块的模板，根据K个初始预测模式，确定当前块的权重导出模式。以VVC为例，GPM有64种权重导出模式，解码端通过对模板进行分析来估计哪些权重导出模式被选中的可能性大，哪些权重导出模式被选中的可能性小。以帧间的GPM为例，一个典型的场景是2个物体相互运动，GPM可以用于2个物体的边缘。假设当前块与模板中都包含物体的边缘，那么就可以用模板上的边缘来推测当前块的边缘。一个例子如下，给定2个运动信息，根据这2个运动信息分别得到2个模板上的预测值，记为第一模板预测值和第二模板预测值，根据某一权重导出模式导出模板权重，根据第一模板预测值，第二模板预测值和模板权重，确定该权重导出模式对应的模板预测值，因为模板的重建值是可得的，所以根据该权重导出模式对应的模板预测值和模板的重建值可以得到该权重导出模式在模板上的代价。这样可以根据各个权重导出模式在模板上的代价估计它被选中的概率。一种是根据各个权重导出模式在模板上的代价对权重导出模式进行排序。代价小的模式在前，代价大的模式在后。因为认为经过排序的权重导出模式大致是从概率大到概率小排列的，所以可以使用变长编码的方法来处理。一种可能的实现方式是，构建一个权重导出模式的列表，编码端把当前块选择的当前块的权重导出模式按列表中的索引写入码流中，解码器解析码流中的索引，从列表中确定当前块的权重导出模式。
由上述可知，本申请实施例在根据K个初始预测模式，确定当前块的权重导出模式过程中，需要根据权重导出模式，确定模板的权重。
下面对本申请实施例涉及的根据权重导出模式，确定模板权重的过程进行介绍。
模板匹配是利用相邻像素之间的相关性，把当前块周边的一些区域作为模板。在当前块进行编解码时，按照编码顺序其左侧及上侧已经解码完成。在帧间预测时，在参考帧中寻找模板的最佳匹配位置从而确定当前块的运动信息或者说运动矢量。在帧内预测时，利用模板来确定当前块的帧内预测模式。
本申请对当前块的模板的具体形状不做限制。
在一些实施例中，模板包括当前块的上方已解码区域、左侧已解码区域和左上方已解码区域中的至少一个。
可选的，上方已解码区域的宽度与当前块的宽度相同，左侧已解码区域的高度与当前块的高度相同，左上方已解码区域的宽度与左侧已解码区域的宽度相同，左上方已解码区域的高度与上方已解码区域的高度相同。
目前，如上述表5所示，以K＝2为例，K个预测模式包括第一预测模式和第二预测模式，第一预测模式和第二预测模式对应的模板为当前块的上方已解码区域，或者为当前块左侧已解码区域，或者为当前块的左侧已解码和上方已 解码区域。例如图16所示，以GPM的索引为2的权重导出模式为例，当前块的权重矩阵的白色区域为第一预测模式的预测值对应的权重，黑色区域为第二预测模式的预测值对应的权重。如图16所示，第一预测模式对应的第一模板为当前块的上方已解码区域，第二预测模式对应的第二模板为当前块的左侧已解码区域，但是，与第二预测模式靠近的模板包括左侧区域外，还包括部分上方已解码区域，因此，现有技术对于模板的划分不够精细，进而无法基于模板准确出当前块的权重导出模式。
为了解决该技术问题，本申请可以通过权重导出模式实现模板的更精细划分，例如图17A所示，本申请将当前块的权重导出模式对应的权重矩阵的分割线向模板区域延伸，以将模板进行划分，例如将分割线一侧的模板记为第一模板，将分割线另一侧的模板记为第二模板。第一模板对应第一预测模式，第二模板对应的第二预测模式，第一模板的权重根据第一预测模式对应的权重确定，第二模板的权重根据第二预测模式对应的权重确定，例如第一模板的权重与第一预测模式对应的权重相同，第二模板的权重与第二预测模式对应的权重相同。
在一些实施例中，根据上述方法划分的第一模板和第二模板可能不是长方形，例如图17A所示，第一模板和第二模板具有斜边，对不规则的模板计算代价较复杂。
为了降低模板匹配的复杂度，在一些实施例中，可以直接将权重矩阵向模板区域延伸，例如向左向上延伸，将权重矩阵覆盖到模板上面，从而确定出模板的权重来。例如图17B所示，可以选择把当前块左上侧的区域的矩形区域加入到模板中，将模板和当前块拼起来构成一个矩形。当然也可以只使用左侧部分和上方部分作为模板。如图17B所示，这里以加入左上侧的区域举例，左侧、左上侧和上侧的倒L形框中的区域为模板区域，右下方的矩形区域为当前块，那么向左上方扩展出来的权重矩阵就变成了模板的权重矩阵。这样不需要真正地划分模板，而是利用权重模拟划分模板。可以说是利用权重导出模式模拟划分模板，或者说利用权重导出模式确定模板权重，根据模板权重确定模板匹配时的代价。由于不同的块形状下模板的划分可能不同，而本申请则不需要对各种形状的块设置各种规则，例如不需要设置表5所示的预测模式与模板的对应关系。而就“划分”的准确性来说，本申请的模拟划分也比只区分左侧区域和上方区域的方法更精细和准确。
在一些实施例中，可以将图17B所示的当前块的左侧区域和上方区域作为当前块的模板，而不考虑左上方的矩阵区域。
在模板匹配时，可以只使用一个分量(Y、Cb、Cr或R、G、B)，比如在YUV格式中可以使用Y，因为Y占主导地位，当然也可以所有分量都使用。为了便于描述，本申请以一个分量为例进行说明，例如以Y分量进行模板匹配为例，其他分量参照即可。
本申请实施例中，根据权重导出模式导出模板权重可以与预测值的权重的导出过程合并，例如同时导出模板权重和预测值的权重。其中预测值的权重可以理解为预测值对应的权重，例如使用第一预测模式得到第一预测值，使用第二预测模式得到第二预测值，根据权重导出模式确定第一预测值的第一权重，根据权重导出模式确定第二预测值的第二权重，将第一预测值与第一权重的乘积，以及第二预测值与第二权重的乘积的和值，确定为当前块的预测值。
本申请为了与模板权重区分，将上的第一权重和第二权重称为预测值的权重。
在一些实施例中，上述根据权重导出模式，确定模板权重包括如下步骤：
步骤A、根据权重导出模式，确定角度索引和距离索引；
步骤B、根据角度索引、距离索引和模板的大小，确定模板权重。
本申请可以采用与导出预测值的权重相同的方式，导出模板权重。例如，首先根据权重导出模式，确定角度索引和距离索引，其中角度索引可以理解为权重导出模式导出的各权重的分界线角度索引。示例性的，可以根据上述表2，确定出权重导出模式对应的角度索引和距离索引，例如权重导出模式为27，则对应的角度索引为12，距离索引为3。接着，根据角度索引、距离索引和模板的大小，确定出模板权重。
上述步骤B中根据权重导出模式导出模板权重的方式包括但不限于如下几种方式：
方式一，根据角度索引、距离索引和模板的大小，直接确定模板权重。此时，上述步骤B包括如下步骤B11至步骤B13的步骤：
步骤B11、根据角度索引、距离索引和模板的大小，确定模板中像素点的第一参数；
步骤B12、根据模板中像素点的第一参数，确定模板中像素点的权重；
步骤B13、根据模板中像素点的权重，确定模板权重。
本实现方式中，根据角度索引、距离索引、模板的大小和当前块的大小，确定模板中像素点的权重，进而将模板中每个点的权重组成的权重矩阵，确定为模板权重。
本申请的第一参数用于确定权重。在一些实施例中，第一参数也称为权重索引。
在一种可能的实现方式中，可以根据如下方式确定出偏移量和第一参数：
模板权重导出过程的输入有：如图18所示，当前块的宽度nCbW，当前块的高度nCbH；左侧模板的宽度nTmW，上侧模板的高度nTmH；GPM的“划分”角度索引变量angleId；GPM的距离索引变量distanceIdx；分量索引变量cIdx，示例性的，本申请以亮度分量为例，因此cIdx为0，表示亮度分量。
其中，变量nW,nH,shift1,offset1,displacementX,displacementY,partFlip还有shiftHor按如下方法导出：
nW＝(cIdx＝＝0)？nCbW:nCbW*SubWidthC
nH＝(cIdx＝＝0)？nCbH:nCbH*SubHeightC
shift1＝Max(5,17-BitDepth)其中BitDepth是编解码的比特深度
offset1＝1<<(shift1-1)
displacementX＝angleIdx
displacementY＝(angleIdx+8)％32
partFlip＝(angleIdx>＝13&&angleIdx<＝27)？0:1
shiftHor＝(angleIdx％16＝＝8||(angleIdx％16！＝0&&nH>＝nW))？0:1
其中，偏移量offsetX and offsetY按如下方法导出：
–如果shiftHor的值为0:
offsetX＝(-nW)>>1
offsetY＝((-nH)>>1)+(angleIdx<16？(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3))
–否则(即shiftHor的值为1):
offsetX＝((-nW)>>1)+(angleIdx<16？(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)
offsetY＝(-nH)>>1
模板权重矩阵wTemplateValue[x][y](其中x＝-nTmW..nCbW–1，y＝-nTmH..nCbH-1，去除x，y同时大于等于0的情况)注意这个例子里面以当前块的左上角坐标为(0，0)按如下方法导出:
–变量xL和yL按如下方法导出:
xL＝(cIdx＝＝0)？x:x*SubWidthC
yL＝(cIdx＝＝0)？y:y*SubHeightC
其中disLut按上述表3确定
其中，第一参数weightIdx按如下方法导出：
weightIdx＝(((xL+offsetX)<<1)+1)*disLut[displacementX]+
(((yL+offsetY)<<1)+1)*disLut[displacementY]
根据上述方法，确定出第一参数weightIdx后，根据weightIdx确定出模板中点(x，y)处的权重。
本申请中，上述步骤B12中根据模板中像素点的第一参数，确定模板中像素点的权重的方式包括的不限于如下几种：
方式1，根据模板中像素点的第一参数，确定模板中像素点的第二参数；根据模板中像素点的第二参数，确定模板中像素点的权重。
其中第二参数也用于确定权重。在一些实施例中，上述第二参数也称为第一分量下的权重索引，该第一分量可以为亮度分量、色度分量等。
例如，根据公式，确定出模板中像素点的权重：
weightIdxL＝partFlip？32+weightIdx:32-weightIdx
wTemplateValue[x][y]＝Clip3(0,8,(weightIdxL+4)>>3)
其中，wTemplateValue[x][y]为模板中点(x，y)的权重，weightIdxL为模板中点(x，y)的第二参数，也称为在第一分量(例如亮度分量)下的权重索引，wTemplateValue[x][y]为模板中点(x，y)的权重，partFlip为中间变量，根据角度索引angleIdx确定，例如上述所述：partFlip＝(angleIdx>＝13&&angleIdx<＝27)？0:1，也就是说，partFlip的值为1或0，当partFlip为0时，weightIdxL为32–weightIdx，当partFlip为1时，weightIdxL为32+weightIdx，需要说明的是，这里的32只是一种示例，本申请不局限于此。
方式2，根据模板中像素点的第一参数、第一预设值和第二预设值，确定模板中像素点的权重。
为了降低模板权重的计算复杂度，在方式2中将模板中像素点的权重限定为第一预设值或第二预设值，也就是说，模板中像素点的权重要么为第一预设值，要么是第二预设值，进而降低模板权重的计算复杂度。
本申请对第一预设值和第二预设值的具体取值不做限制。
可选的，第一预设值为1。
可选的，第二预设值为0。
在一种示例，可以通过如下公式确定出模板中像素点的权重：
wTemplateValue[x][y]＝(partFlip？weightIdx:-weightIdx)>0？1:0
其中，wTemplateValue[x][y]为模板中点(x，y)的权重，上述“1:0”中的1为第一预设值，0为第二预设值。
在上述方式一中，通过权重导出模式确定出模板中每个点的权重，模板中每个点的权重组成的权重矩阵作为模板权重。
方式二，根据权重导出模式确定出当前块和模板的权重，也就是说，在该方式二中，将当前块和模板组成的合并区域作为一个整体，根据权重导出模式导出合并区域中像素点的权重。基于此上述步骤B包括如下步骤B21和S102-B22步骤：
步骤B21、根据角度索引、距离索引、模板的大小和当前块的大小，确定当前块和模板组成的合并区域中像素点的权重；
步骤B22、根据模板的大小和合并区域中像素点的权重，确定模板权重。
在该方式二中，将当前块和模板作为一个整体，根据角度索引、距离索引、模板的大小和当前块的大小，确定当前块和模板组成的合并区域中像素点的权重，进而根据模板的大小，将合并区域中像素点的权重中模板对应的权重确定为模板权重，例如图18所示，将合并区域中L型模板区域对应的权重确定为模板权重。
该方式二中，在一次权重确定过程中，不仅确定出模板权重，并且确定出预测值的权重，即合并区域对应的权重中除去模板权重后，剩余的为预测值的权重，进而可以根据预测值的权重执行后续的预测过程，无需重新确定预测值的权重，进而减少了预测步骤，提升了预测效率。
本申请对上述根据角度索引、距离索引、模板的大小和当前块的大小，确定当前块和模板组成的合并区域中像素点的权重的具体实现方式不做限制。
在一些实施例中，上述步骤B21中确定合并区域中像素点的权重包括如下步骤B211至步骤B212的步骤：
步骤B211、根据角度索引、距离索引和合并区域的大小，确定合并区域中像素点的第一参数；
步骤B212、根据合并区域中像素点的第一参数，确定合并区域中像素点的权重。
本实现方式中，根据角度索引、距离索引、合并区域的大小，确定合并区域中像素点的权重，合并区域中每个点的权重组成权重矩阵。
在一种可能的实现方式中，可以根据如下方式确定出偏移量和第一参数：
合并区域权重导出过程的输入有：如图18所示，当前块的宽度nCbW，当前块的高度nCbH；左侧模板的宽度nTmW，上侧模板的高度nTmH；GPM的“划分”角度索引变量angleId；GPM的距离索引变量distanceIdx；分量索引变量cIdx，示例性的，本申请以亮度分量为例，因此cIdx为0，表示亮度分量。
变量nW,nH,shift1,offset1,displacementX,displacementY,partFlip还有shiftHor按如下方法导出:
nW＝(cIdx＝＝0)？nCbW:nCbW*SubWidthC
nH＝(cIdx＝＝0)？nCbH:nCbH*SubHeightC
shift1＝Max(5,17-BitDepth)其中BitDepth是编解码的比特深度
offset1＝1<<(shift1-1)
displacementX＝angleIdx
displacementY＝(angleIdx+8)％32
partFlip＝(angleIdx>＝13&&angleIdx<＝27)？0:1
shiftHor＝(angleIdx％16＝＝8||(angleIdx％16！＝0&&nH>＝nW))？0:1
其中，偏移量offsetX and offsetY按如下方法导出:
–如果shiftHor的值为0:
offsetX＝(-nW)>>1
offsetY＝((-nH)>>1)+(angleIdx<16？(distanceIdx*nH)>>3:-((distanceIdx*nH)>>3))
–否则(即shiftHor的值为1):
offsetX＝((-nW)>>1)+(angleIdx<16？(distanceIdx*nW)>>3:-((distanceIdx*nW)>>3)
offsetY＝(-nH)>>1
合并区域中像素点的权重wValueMatrix[x][y](其中x＝-nTmW..nCbW–1，y＝-nTmH..nCbH-1)注意这个例子里面以当前块的左上角坐标为(0，0)按如下方法导出:
–变量xL和yL按如下方法导出:
xL＝(cIdx＝＝0)？x:x*SubWidthC
yL＝(cIdx＝＝0)？y:y*SubHeightC
其中disLut按表3确定。
其中，第一参数weightIdx按如下方法导出：
weightIdx＝(((xL+offsetX)<<1)+1)*disLut[displacementX]+
(((yL+offsetY)<<1)+1)*disLut[displacementY]
根据上述方法，确定出第一参数weightIdx后，根据weightIdx确定出合并区域中点(x，y)处的权重。
本申请中，上述步骤B212中根据合并区域中像素点的第一参数，确定合并区域中像素点的权重的方式包括的不限于如下几种：
方式1，根据合并区域中像素点的第一参数，确定合并区域中像素点的第二参数；根据合并区域中像素点的第二参数，确定合并区域中像素点的权重。
例如，根据公式，确定出合并区域中像素点的权重：
weightIdx＝(((xL+offsetX)<<1)+1)*disLut[displacementX]+
(((yL+offsetY)<<1)+1)*disLut[displacementY]
weightIdxL＝partFlip？32+weightIdx:32-weightIdx
wValueMatrix[x][y]＝Clip3(0,8,(weightIdxL+4)>>3)
其中，wValueMatrix[x][y]为合并区域中点(x，y)的权重，weightIdxL为合并区域中点(x，y)的第二参数，wTemplateValue[x][y]为合并区域中点(x，y)的权重。
方式2，根据合并区域中像素点的第一参数、第一预设值和第二预设值，确定合并区域中像素点的权重。
示例性的，合并区域中像素点的权重为第一预设值或为第二预设值。
为了降低合并区域权重的计算复杂度，在方式2中将合并区域中像素点的权重限定为第一预设值或第二预设值，也就是说，合并区域中像素点的权重要么为第一预设值，要么是第二预设值，进而降低合并区域权重的计算复杂度。
本申请对第一预设值和第二预设值的具体取值不做限制。
可选的，第一预设值为1。
可选的，第二预设值为0。
在一种示例，可以通过如下公式确定出合并区域中像素点的权重：
wTemplateValue[x][y]＝(partFlip？weightIdx:-weightIdx)>0？1:0
其中，wValueMatrix[x][y]为合并区域中点(x，y)的权重，上述“1:0”中的1为第一预设值，0为第二预设值。
在上述方式二中，通过权重导出模式确定出当前块和模板组成的合并区域中每个点的权重，进而根据模板的大小，将合并区域中模板对应的权重确定为模板权重，且该方式二中，同时确定出预测值的权重，无需执行后续的预测值的权重的确定步骤，进行降低了预测步骤，提升了预测效率。
上述对根据权重导出模式，确定模板权重的过程进行详细介绍，其中上述权重导出模式可以为本申请实施例涉及的任意一个权重导出模式，例如为当前块的权重导出模式，或者为N个候选预测模式中的第i个候选预测模式等等。
S103、根据当前块的权重导出模式，确定当前块的预测值。
本申请中，S103中解码端根据当前块的权重导出模式，确定当前块的预测值的方式包括但不限于如下几种：
方式一，解码端根据当前块的权重导出模式和K个初始预测模式，确定当前块的预测值，具体的，解码端根据当前块的权重导出模式确定预测值的权重，再根据K个初始预测模式对当前块进行预测，得到K个预测值，然后利用预测值的权重对K个预测值进行加权平均计算，得到当前块的预测值。
需要说明的是，上述根据当前块的权重导出模式导出的预测值的权重可以理解为导出当前块中每一个像素点对应的权重，也可以理解为导出当前块对应的权重矩阵。其中，基于预测值的权重确定当前块的预测值时，可以是确定当前块中每一个像素点对应的K个预测值，并根据每一个像素点对应的K个预测值和预测值的权重，确定出每个像素点对应的预测值，当前块中每一个像素点对应的预测值构成当前块的预测值。可选的，基于预测值的权重确定当前块的预测值时，还可以是按照块来执行，例如，确定当前块的K个预测值，根据当前块的权重矩阵对当前块的K个预测值进行加权，得到当前块的预测值。
方式二，解码端可以使用当前块的权重导出模式对上述K个初始预测模式进行改善，此时，上述S103包括如下S103-A和S103-B的步骤：
S103-A、根据当前块的权重导出模式，确定K个调整的预测模式；
S103-B、根据K个调整的预测模式和当前块的权重导出模式，确定当前块的预测值。
本申请实施例中，可以认为GPM有K+1个要素，即一个权重导出模式和K个预测模式。由于一个权重导出模式和K个预测模式共同产生一个预测块，这个预测块作用于当前块。它们之间是有关联的。举几个例子，比如当前块包含2个相对运动的物体的边缘，这是帧间GPM的一个理想的场景。那么理论上这个“划分”应该发生在物体的边缘，但是实际上“划分”有有限种可能，不可能覆盖任意的边缘，有时会选择相近的“划分”，那这样可能相近的“划分”不止一种，选择哪种就取决于哪种“划分”与2个预测模式组合的结果最优。同理，选择哪种预测模式有时同样取决于哪种组合的结果最优，因为即使是在使用该预测模式的部分，对自然视频而言，这部分也很难和当前块完全匹配，最终选择的可能是编码效率最高的。另一种GPM用得比较多的地方是当前块包含一个物体中存在相对运动的部分。比如说胳膊的摆动等导致扭曲、变形的地方，这样的“划分”就更模糊，可能最终取决于哪种组合的结果最优。还有一种场景是帧内的预测，自然图像中某些部分的纹理是很复杂的，某些部分存在一种纹理向另一种纹理的渐变，某些部分可能无法用简单的一个方向来表述，帧内GPM可以提供出更复杂的预测块，而帧内编码的块相对于相同量化下的帧间编码的块通常存在更大残差，选择哪种预测模式可能最终取决于哪种组合的结果最优。
可见K+1个要素之间是会互相影响的，因而可以利用权重导出模式确定预测模式，也可以利用预测模式确定权重导出模式。更进一步地，可以看到某些模式信息也是可以进行改善(refine)的。所谓改善，就是在一个初始的模式信息的基础上，通过采集更多的相关信息，使用更多的计算等手段，获取一个更好的模式信息。一般说的refine是解码端的操作。而基于模板也可以对运动信息进行改善refine。即在初始的运动信息的基础上，在解码端进行一个小范围的搜索，根据所搜索的运动信息与模板的匹配程度来优化运动信息。所谓运动信息与模板的匹配程度一种理解是运动信息在模板上的预测值与模板的重建值的匹配程度。
对应到GPM，可以先根据预测模式确定权重导出模式，再根据权重导出模式refine预测模式。或者先根据权重导出模式确定预测模式，再返回来根据预测模式refine权重导出模式。当然二者也可以交替实施，如果复杂度允许。模板可以作为实施这些方法的纽带，更进一步地，模板权重可以作为实施这些方法的纽带。即根据预测模式在模板上的预测值，以及权重导出模式对应的子模板或模板权重，来计算模板上的代价。利用模板上的代价来推导或者说确定预测模式或权重导出模式。
也就是说，在本申请实施例中，根据K个初始预测模式，确定当前块的权重导出模式，再使用当前块的权重导出模式对K个初始预测模式进行改善，得到K个调整的预测模式，进而根据当前块的权重导出模式和K个调整的预测模式，确定当前块的预测值。
下面通过具体的实施例，对上述S102中根据K个初始预测模式确定当前块的权重导出模式的过程，以及上述S103-A中根据当前块的权重导出模式，确定K个调整的预测模式的过程进行详细介绍。
在一些实施例中，上述S102包括如下S102-A至S102-C的步骤：
S102-A、确定N个候选权重导出模式，N为大于1的正整数；
S102-B、针对N个候选权重导出模式中的第i个候选权重导出模式，根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价，i为从1到N的正整数；
S102-C、根据N个候选权重导出模式分别对应的第一代价，确定当前块的权重导出模式。
在该实施例中，解码端首先确定N个候选权重导出模式，该N个候选权重导出模式构成权重导出模式的列表，解码端通过计算这N个候选权重导出模式中每个候选权重导出模式对应的第一代价，根据第一代价，从这N个候选权重导出模式中，确定出当前块的权重导出模式。
例如，将N个候选权重导出模式中第一代价最小的候选权重导出模式，确定为当前块的权重导出模式。
再例如，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表；解码码流，得到权重导出模式索引；将候选权重导出模式列表中权重导出模式索引对应候选权重导出模式，确定为当前块的权重导出模式。
可选的，上述第一代价可以为SAD、SATD或SEE等代价。
解码端确定N个候选权重导出模式中每个候选权重导出模式对应的第一代价的过程相同，为了便于描述，以确定N个候选权重导出模式中的第i个候选权重导出模式对应的第一代价为例。
上述S102-B中根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价的方式包括但不限于如下几种：
方式一，使用第i个候选权重导出模式和K个初始预测模式对当前块的模板进行预测，得到模板的第一预测值；根据模板的第一预测值和模板的重建值，确定第i个权重导出模式对应的第一代价。
具体的，根据第i个候选权重导出模式确定模板权重，使用K个初始预测模式对模板进行预测，得到模板的K个 预测值，使用模板权重对模板对应的K个预测值进行加权处理，得到模板的预测值，为了便于描述，将该模板的预测值记为模板的第一预测值。接着，根据模板的第一预测值和模板的重建值，确定该第i个候选权重导出模式关于模板的预测代价，将该预测代价记为第i个权重导出模式对应的第一代价。
需要说明的是，根据根据第i个候选权重导出模式确定模板权重的具体过程可以参照上述根据权重导出模式，确定模板权重的具体内容描述，在此不再赘述。
根据上述方法，可以确定出N个候选权重导出模式中每个候选权重导出模式对应的第一代价，进而将N个候选权重导出模式中第一代价最小的候选权重导出模式确定为当前块的权重导出模式，或者，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表，将候选权重导出模式列表中权重导出模式索引对应候选权重导出模式，确定为当前块的权重导出模式。
在该方式一的基础上，上述S103-A中根据当前块的权重导出模式，确定K个调整的预测模式包括如下步骤：
S103-A1、根据当前块的权重导出模式，对K个初始预测模式进行改善，得到K个调整的预测模式。
其中，S103-A1的实现方式包括但不限于如下几种示例：
示例1，确定与当前块的权重导出模式相关的至少一个预测模式，这至少一个预测模式可以是预测角度与当前块的权重导出模式的分界线平行或近似平行的预测模式或者为预测角度与当前块的权重导出模式的分界面垂值或近似垂直的预测模式。将K个初始预测模式中的至少一个初始预测模式，使用与当前块的权重导出模式相关的至少一个预测模式中的预测模式进行替换，形成多个新的预测模式组合，每个新的预测模式组合中包括K个预测模式，针对每个新的预测模式组合，使用该新的预测模式组合所包括的K个预测模式和当前块的权重导出模式对目标进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该新的预测模式组合对应的代价，进而可以获得上述形成的多个新的预测模式组合中每一个新的预测模式组合对应的代价。同理，使用当前块的权重导出模式和K个初始预测模式对当前块进行预测，确定K个初始预测模式对应的代价。这样，解码端可以根据新的预测模式组合对应的代价，以及K个初始预测模式对应的代价，得到K个调整的预测模式，例如若新的预测模式组合1对应的代价最小，则将该新的预测模式组合1所包括的K个预测模式，确定为K个调整的预测模式，该新的预测模式组合1中可能包括至少一个初始预测模式，也可能不包括初始预测模式。
示例2，通过如下S103-A11至S103-A14的步骤对K个初始预测模式进行改善，得到K个调整的预测模式：
S103-A11、根据当前块的权重导出模式，确定第一模板权重；
其中，根据当前块的权重导出模式，确定第一模板权重的具体过程可以参照上述模板权重的导出实施例的具体描述，在此不再赘述。此时，上述模板权重导出实施例中的权重导出模式为当前块的权重导出模式，对应的模板权重为上述第一模板权重。
S103-A12、针对K个初始预测模式中的第j个初始预测模式，确定第j个初始预测模式对应的M个第一候选预测模式，M为大于1的正整数，j为从1到K的正整数。
需要说明的是，上述S103-A12与上述S103-A11在执行过程中没有先后顺序，即上述S103-A12可以在上述S103-A11之后执行，也可以在S103-A11之前执行，或者与上述S103-A11同步执行，本申请实施例对此不做限制。
在该示例中，解码端使用当前块的权重导出模式对K个初始预测模式中每个初始预测模式进行改善，得到K个初始预测模式中每个初始预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
其中，确定K个初始预测模式中每个初始预测模式对应的调整的预测模式的过程相同，为了便于描述，本申请实施例以确定K个初始预测模式中第j个初始预测模式对应的调整的预测模式为例进行说明。
本申请实施例中，解码端针对第j个初始预测模式，首先确定该第j个初始预测模式对应的M个第一候选预测模式。
解码端确定第j个初始预测模式对应的M个第一候选预测模式的方式包括如下两种情况：
情况1，若第j个初始预测模式为帧内预测模式，将与第j个初始预测模式相似的M个帧内预测模式，确定为M个第一候选预测模式。例如，第j个初始预测模式为角度预测模式，则将与第j个初始预测模式的角度相似(或相近)的M个角度预测模式，确定第j个初始预测模式对应的M个第一候选预测模式。
情况2，若第j个初始预测模式为帧间预测模式，则根据第j个初始预测模式进行搜索，得到M个第一运动信息，根据这M个第一运动信息，确定第j个预测模式对应的M个第一候选预测模式。例如，对于第j个初始预测模式，其运动矢量是(xInit，yInit)，设置一个搜索范围如水平方向从xInit-sR到xInit+sR，竖直方向从yInit-sR到yInit+sR的矩形区域，其中sR可以是2，4，8等。该矩形区域内的每一个运动矢量都可以和第j个初始预测模式的其他信息，如参考帧索引和预测列表标志等组合起来确定一个运动信息，从而确定一个预测模式，这样可以获得第j个预测模式对应的M个第一候选预测模式。
在一些实施例中，上述根据第j个初始预测模式进行搜索，得到M个第一运动信息的一种方式为，根据第j个初始预测模式对应的运动矢量，在预设搜索范围内进行搜索，得到M个偏移量，每个偏移量包括第一方向(例如x方向)上的偏移量和第二方向(例如y方向)上的偏移量。进而，根据这M个偏移量，得到M个第一运动信息，例如，对于M个偏移量中的每个偏移量，根据该偏移量对第j个初始预测模式对应的运动矢量进行偏移，得到一个第一运动信息。
S103-A13、根据第一模板权重，确定使用M个第一候选预测模式分别对模板进行预测时所对应的第二代价。
根据上述情况1或情况2的方法，确定出第j个初始预测模式对应的M个第一候选预测模式后，确定这M个第一候选预测模式对应的代价，进而根据代价确定第j个初始预测模式对应的调整的预测模式。为了便于描述，将第一 候选预测模式对应的代价，记为第二代价。
具体的，针对M个第一候选预测模式中的每个第一候选预测模式，使用该第一候选预测模式对当前块的模板进行预测，得到一个预测值，接着，根据该第一模板权重和该预测值确定该第一候选预测模式关于模板的第二代价。
下面以计算M个第一候选预测模式中一个第一候选预测模式关于模板的第二代价为例进行说明。
为了便于描述，以模板中的第i个像素点为例，该第i个像素点可以理解为模板中的任意一个点，也就是说，确定模板中每个点的第二代价的过程相同，参照第i个像素点即可。具体是，使用该第一候选预测模式对模板进行预测，得到模板的预测值，将第i个像素点在模板的预测值中对应的预测值记为第i个预测值，将该第i个像素点在模板的重建值中对应的重建值记为第i个重建值，进而根据第i个预测值和第i个重建值，以及第i个像素点在第一模板权重中的权重，确定该第一候选预测模式在第i个像素点的第二代价。根据上述方法，确定出该第一候选预测模式在模板中的每个点或多个点处的第二代价，进而根据模板中每个点或多个点的第二代价，确定出第一候选预测模式关于模板的第二代价。例如，将该第一候选预测模式在模板中像素点处的第二代价之和，确定为该第一候选预测模式关于模板的第二代价。
示例性的，以SAD代价为例，可以根据如下公式(1)确定出第一候选预测模式在模板中的第i个像素点(x，y)处的第二代价：
tempValueA[x][y]＝abs(predTemplateSamplesCandA[x][y]-recTemplateSamples[x][y])*wTemplateValue[x][y]1    (1)
示例性的，根据如下公式(2)确定出第一候选预测模式关于模板的第二代价：
costCandA＝∑tempValueA[x][y]     (2)
其中，abs(predTemplateSamplesCandA[x][y]-recTemplateSamples[x][y])是模板中点(x，y)的预测值predTemplateSamplesCandA和重建值recTemplateSamples的差的绝对值，将该差的绝对值称为点(x，y)对应的第二代价。将这个值乘以该点在第一模板权重中的权重值wTemplateValue[x][y]，得到tempValueA[x][y]。tempValueA[x][y]可以认为是该第一候选预测模式在这个点(x，y)处的第二代价。该第一候选预测模式在模板上的总的第二代价costCandA为模板上每一个点的第二代价累加。
需要说明的是，上述以SAD为例来确定第一候选预测模式关于模板的第二代价，可选的还可以根据SATD、MSE等代价计算方法，确定第一候选预测模式关于模板的第二代价。
根据上述方法，可以确定出M个第一候选预测模式中每个第一候选预测模式对应的第二代价，接着，执行如下S103-A14。
S103-A14、根据M个第一候选预测模式分别对应的第二代价，确定第j个初始预测模式对应的调整的预测模式。
例如，将M个第一候选预测模式中第二代价最小的第一候选预测模式，确定为第j个初始预测模式对应的调整的预测模式。
上述实施例以确定K个初始预测模式中的第j个初始预测模式对应的调整的预测模式为例进行说明，参照上述描述，可以确定出K个初始预测模式中每个初始预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
在该方式一中，解码端首先根据K个初始预测模式，确定出当前块的权重导出模式，再根据当前块的权重导出模式对K个初始预测模式进行改善，例如通过上述示例1或示例2的方式进行改善，得到K个调整的预测模式，进而使用当前块的权重导出模式和K个调整的预测模式对当前块进行预测，得到当前块的预测值。
本申请实施例中，解码端除了使用上述方式一的方式确定当前块的权重导出模式和K个调整的预测模式外，还可以通过如下方式二，确定当前块的权重导出模式和K个调整的预测模式。
方式二，将预测模式的改善和当前块的权重导出模式的确定融合在一起，同步进行，此时上述S102-B中根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价包括如下S102-B11至S102-B12的步骤：
S102-B11、根据第i个候选权重导出模式，对K个初始预测模式进行改善，得到第i个候选权重导出模式对应的K个第一改善预测模式。
在该方式二中，对于N个候选预测模式中的第i个候选权重导出模式，首先使用该第i个候选权重导出模式对K个初始预测模式进行改善，得到该第i个候选权重导出模式对应的K个改善预测模式，为了便于描述，将此处第i个候选权重导出模式对应的K个改善预测模式记为K个第一改善预测模式。接着该第i个候选权重导出模式，以及该第i个候选权重导出模式对应的K个第一改善预测模式，确定第i个候选权重导出模式对应的第一代价，这样可以得到更精确的当前块的权重导出模式和K个调整的预测模式。
其中，S102-B11的实现方式包括但不限于如下几种示例：
示例1，确定与第i个候选权重导出模式相关的至少一个预测模式，这至少一个预测模式可以是预测角度与第i个候选权重导出模式的分界线平行或近似平行的预测模式或者为预测角度与第i个候选权重导出模式的分界面垂值或近似垂直的预测模式。将K个初始预测模式中的至少一个初始预测模式，使用与第i个候选权重导出模式相关的至少一个预测模式中的预测模式进行替换，形成多个新的预测模式组合，每个新的预测模式组合中包括K个预测模式，针对每个新的预测模式组合，使用该新的预测模式组合所包括的K个预测模式和第i个候选权重导出模式对模板进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该新的预测模式组合对应的代价，进而可以获得上述形成的多个新的预测模式组合中每一个新的预测模式组合对应的代价。同理，使用第i个候选权重导出模式和K个初始 预测模式对当前块进行预测，确定K个初始预测模式对应的代价。这样，解码端可以根据新的预测模式组合对应的代价，以及K个初始预测模式对应的代价，得到K个第一改善预测模式，例如若新的预测模式组合1对应的代价最小，则将该新的预测模式组合1所包括的K个预测模式，确定为K个第一改善预测模式，该新的预测模式组合1中可能包括至少一个初始预测模式，也可能不包括初始预测模式。
示例2，通过如下S102-B11-1至S102-B11-4的步骤对对K个初始预测模式进行改善，得到第i个候选权重导出模式对应的K个第一改善预测模式：
S102-B11-1、根据第i个候选权重导出模式，确定第二模板权重。
其中，根据第i个候选权重导出模式，确定第二模板权重的具体过程可以参照上述模板权重的导出实施例的具体描述，在此不再赘述。此时，权重导出模式为第i个候选权重导出模式，对应的模板权重为第二模板权重。
S102-B11-2、针对K个初始预测模式中的第j个初始预测模式，确定第j个初始预测模式对应的M个第一候选预测模式，M为正整数，j为从1到K的正整数。
需要说明的是，上述S102-B11-2与上述S102-B11-1在执行过程中没有先后顺序，即上述S102-B11-2可以在上述S102-B11-1之后执行，也可以在S102-B11-1之前执行，或者与上述S102-B11-1同步执行，本申请实施例对此不做限制。
在该示例中，解码端使用第i个候选权重导出模式对K个初始预测模式中每个初始预测模式进行改善，得到第i个候选权重导出模式下，K个初始预测模式中每个初始预测模式对应的第一改善预测模式，进而得到K个第一改善预测模式。
其中，确定K个初始预测模式中每个初始预测模式对应的第一改善预测模式的过程相同，为了便于描述，本申请实施例以确定K个初始预测模式中第j个初始预测模式对应的第一改善预测模式为例进行说明。
本申请实施例中，解码端针对第j个初始预测模式，首先确定该第j个初始预测模式对应的M个第一候选预测模式。
解码端确定第j个初始预测模式对应的M个第一候选预测模式的方式包括如下两种情况：
情况1，若第j个初始预测模式为帧内预测模式，将与第j个初始预测模式相似的M个帧内预测模式，确定为M个第一候选预测模式。
情况2，若第j个初始预测模式为帧间预测模式，则根据第j个初始预测模式进行搜索，得到M个第一运动信息，根据这M个第一运动信息，确定第j个预测模式对应的M个第一候选预测模式。
在一些实施例中，上述根据第j个初始预测模式进行搜索，得到M个第一运动信息的一种方式为，根据第j个初始预测模式对应的运动矢量，在预设搜索范围内进行搜索，得到M个偏移量，每个偏移量包括第一方向(例如x方向)上的偏移量和第二方向(例如y方向)上的偏移量。进而，根据这M个偏移量，得到M个第一运动信息，例如，对于M个偏移量中的每个偏移量，根据该偏移量对第j个初始预测模式对应的运动矢量进行偏移，得到一个第一运动信息。
S102-B11-3、根据第二模板权重，确定使用M个第一候选预测模式分别对模板进行预测时所对应的第三代价。
根据上述情况1或情况2的方法，确定出第j个初始预测模式对应的M个第一候选预测模式后，确定这M个第一候选预测模式对应的代价，进而根据代价确定第j个初始预测模式对应的第一改善预测模式。为了便于描述，将此时第一候选预测模式对应的代价，记为第三代价。
具体的，针对M个第一候选预测模式中的每个第一候选预测模式，使用该第一候选预测模式对当前块的模板进行预测，得到一个预测值，接着，根据该第二模板权重和该预测值确定该第一候选预测模式关于模板的第三代价。
下面以计算M个第一候选预测模式中一个第一候选预测模式关于模板的第三代价为例进行说明。
为了便于描述，以模板中的第i个像素点为例，该第i个像素点可以理解为模板中的任意一个点，也就是说，确定模板中每个点的第三代价的过程相同，参照第i个像素点即可。具体是，使用该第一候选预测模式对模板进行预测，得到模板的预测值，将第i个像素点在模板的预测值中对应的预测值记为第i个预测值，将该第i个像素点在模板的重建值中对应的重建值记为第i个重建值，进而根据第i个预测值和第i个重建值，以及第i个像素点在第二模板权重中的权重，确定该第一候选预测模式在第i个像素点的第三代价。根据上述方法，确定出该第一候选预测模式在模板中的每个点或多个点处的第三代价，进而根据模板中每个点或多个点的第三代价，确定出第一候选预测模式关于模板的第三代价。例如，将该第一候选预测模式在模板中像素点处的第三代价之和，确定为该第一候选预测模式关于模板的第三代价。
示例性的，可以参照上述公式(1)和(2)确定出第一候选预测模式关于模板的第三代价，具体是，是将上述公式(1)中wTemplateValue[x][y]替换为像素点(x，y)在第二模板权重上对应的权重值。
根据上述方法，可以确定出M个第一候选预测模式中每个第一候选预测模式对应的第三代价，接着，执行如下S102-B11-4。
S102-B11-4、根据M个第一候选预测模式分别对应的第三代价，确定第j个初始预测模式对应的改善预测模式。
例如，将M个第一候选预测模式中所述第三代价最小的第一候选预测模式，确定为第j个初始预测模式对应的第一改善预测模式。
上述实施例以确定K个初始预测模式中的第j个初始预测模式对应的第一改善预测模式为例进行说明，参照上述 描述，可以确定出K个初始预测模式中每个初始预测模式对应的第一改善预测模式，进而得到第i个候选权重导出模式下的K个第一改善预测模式。
S102-B12、根据第i个候选权重导出模式，以及K个第一改善预测模式，确定第i个候选权重导出模式对应的第一代价。
示例性的，使用第i个候选权重导出模式和K个第一改善预测模式对当前块的模板进行预测，得到模板的第二预测值；根据模板的第二预测值和模板的重建值，确定第i个权重导出模式对应的第一代价。例如，使用第i个候选权重导出模式导出第二模板权重，使用K个第一改善预测模式对模板进行预测，得到K个预测值，使用第二模板权重对这K个预测值进行加权处理，得到模板的第二预测值，根据该模板的第二预测值与模板的重建值，确定第i个权重导出模式对应的第一代价。
上述以确定第i个候选权重导出模式对应的第一代价为例，针对N个候选权重导出模式中的每个候选权重导出模式，均可以采用上述确定第i个权重导出模式对应的第一代价的方式，确定出N个候选权重导出模式中每个候选权重导出模式对应的第一代价，进而根据N个候选权重导出模式中各候选权重导出模式对应的第一代价，确定当前块的权重导出模式。
例如，将N个候选权重导出模式中第一代价最小的候选权重导出模式，确定为当前块的权重导出模式。
再例如，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表；解码码流，得到权重导出模式索引；将候选权重导出模式列表中权重导出模式索引对应候选权重导出模式，确定为当前块的权重导出模式。
在该方式二中，确定出了N个候选权重导出模式中，每个候选权重导出模式对应的K个第一改善预测模式，在此基础上，上述S103-A中根据当前块的权重导出模式，确定K个调整的预测模式包括如下步骤：
S103-A2、将当前块的权重导出模式对应的K个第一改善预测模式，确定为K个调整的预测模式。
在该方式二中，由于将预测模式的改善和当前块的权重导出模式的确定融合在一起，即在确定N个候选权重导出模式对应的第一代价时，确定出了每个候选权重导出模式对应的K个第一改善预测模式。这样，根据第一代价确定出当前块的权重导出模式后，可以将该当前块的权重导出模式对应的K个第一改善预测模式，确定为K个调整的预测模式，进而使用当前块的权重导出模式和这K个调整的预测模式对当前块进行预测，得到当前块的预测值。
该方式二中，对于N个候选权重导出模式中的每个候选权重导出模式，均确定出每个候选权重导出模式对应的K个第一改善预测模式，进而根据每个候选权重导出模式及其对应的K个第一改善预测模式，对模板进行预测，确定第一代价，基于该第一代价可以更精确地从N个候选权重导出模式中确定出当前块的权重导出模式。
本申请实施例中，解码端除了使用上述方式一或方式二的方式确定当前块的权重导出模式和K个调整的预测模式外，还可以通过如下方式三，确定当前块的权重导出模式和K个调整的预测模式。
方式三，为了降低计算复杂度，在该方式三中，在对初始预测模式进行改善时，不使用每一个候选权重导出模式对初始预测模式进行改善，而是使用简单的P个子模板，例如当前块的全部模板、左侧模板、上侧模板，对初始预测模式进行改善，从而减少初始预测模式改善的计算量。此时上述S102-B中根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价包括如下S102-B21至S102-B23的步骤：
S102-B21、针对K个初始预测模式中的第j个初始预测模式，确定第j个初始预测模式关于P个子模板的第二改善预测模式，P为正整数，j为从1到K的正整数。
在该方式三中，解码端使用P个子模板对K个初始预测模式进行改善，得到K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式。对于N个候选权重导出模式中的每个候选权重导出模式，根据该候选权重导出模式与P个子模板的对应关系，从K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式中，得到该候选权重导出模式对应的K个第二改善预测模式，进而根据该候选权重导出模式及其对应的K个第二预测模式对当前块的模板进行预测，确定该候选权重导出模式对应的第一代价。最后，根据N个候选权重导出模式中每个权重导出模式对应的第一代价，确定当前块的权重导出模式。在方式三中，不再单独使用候选预测模式对K个初始预测模式进行改善，进而降低了初始预测模式的改善复杂度。
示例性的，P个子模板包括当前块的左侧模板TM_L、上侧模板TM_A和全部模板TM_AL中的至少一个，其中全部模板TM_AL包括左侧模板TM_L和上侧模板TM_A。
本申请实施例中，解码端确定K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式的过程相同，为了便于描述，本申请实施例以确定K个初始预测模式中第j个初始预测模式关于P个子模板的第二改善预测模式为例进行说明。
在一种示例中，假设P个子模板包括当前块的左侧模板TM_L、上侧模板TM_A和全部模板TM_AL，则确定第j个初始预测模式关于P个子模板的第二改善预测模式包括：确定第j个初始预测模式关于左侧模板TM_L的第二改善预测模式，确定第j个初始预测模式关于上侧模板TM_A的第二改善预测模式，以及确定第j个初始预测模式关于全部模板TM_AL的第二改善预测模式。也就是说，该步骤中，解码端确定第j个初始预测模式关于P个子模板中每个子模板的第二改善预测模式。
本申请实施例对确定第j个初始预测模式关于P个子模板的第二改善预测模式的具体方式不做限制。
在一些实施例中，S102-B21包括如下S102-B21-1至S102-B21-3的步骤：
S102-B21-1、确定第j个初始预测模式对应的M个第一候选预测模式，M为大于1的正整数。
其中，解码端确定第j个初始预测模式对应的M个第一候选预测模式的方式包括如下两种情况：
情况1，若第j个初始预测模式为帧内预测模式，将与第j个初始预测模式相似的M个帧内预测模式，确定为M 个第一候选预测模式。
情况2，若第j个初始预测模式为帧间预测模式，则根据第j个初始预测模式进行搜索，得到M个第一运动信息，根据这M个第一运动信息，确定第j个预测模式对应的M个第一候选预测模式。
在一些实施例中，根据所述第j个初始预测模式进行搜索，得到M个第一运动信息，包括：根据第i个初始预测模式进行搜索，得到M个偏移量；根据M个偏移量，得到M个第一运动信息。
S102-B21-2、针对P个子模板中的第p个子模板，确定使用M个第一候选预测模式分别对第p个子模板进行预测时所对应的第四代价，p为从1到P的正整数。
解码端根据上述情况1或情况2的方法，确定出第j个初始预测模式对应的M个第一候选预测模式后，确定这M个第一候选预测模式分别关于P个子模板中每一个子模板对应的代价，进而根据代价确定第j个初始预测模式关于P个子模板中每一个子模板对应的第二改善预测模式。为了便于描述，将此时第一候选预测模式对应的代价，记为第四代价。
具体的，以P个子模板中的第p个子模板为例，针对M个第一候选预测模式的任一第一候选预测模式，使用该第一候选预测模式对第p个子模板进行预测，得到第p个子模板的预测值；根据第p个子模板的预测值和第p个子模板的重建值，确定第一候选预测模式关于第P个子模板的第四代价。参照该方法，可以确定出M个第一候选预测模式中每一个第一候选预测模式关于P个子模板中每个子模板的第四代价。
举例说明，假设第j个初始预测模式对应3个第一候选预测模式，分别记为第一候选预测模式1、第一候选预测模式2和第一候选预测模式3，假设P个子模板包括左侧模板TM_L、上侧模板TM_A和全部模板TM_AL。首先，对于左侧模板TM_L，使用第一候选预测模式1、第一候选预测模式2和第一候选预测模式3分别对左侧模板TM_L进行预测，得到对应的预测值分别为预测值1、预测值2和预测值3。接着，根据预测值1和左侧模板TM_L的重建值，确定出第一候选预测模式1关于左侧模板TM_L的第四代价，根据预测值2和左侧模板TM_L的重建值，确定出第一候选预测模式2关于左侧模板TM_L的第四代价，根据预测值3和左侧模板TM_L的重建值，确定出第一候选预测模式3关于左侧模板TM_L的第四代价。
同理，对于上侧模板TM_A，使用第一候选预测模式1、第一候选预测模式2和第一候选预测模式3分别对上侧模板TM_A进行预测，得到对应的预测值分别为预测值4、预测值5和预测值6。接着，根据预测值4和左侧模板TM_L的重建值，确定出第一候选预测模式1关于上侧模板TM_A的第四代价，根据预测值5和上侧模板TM_A的重建值，确定出第一候选预测模式2关于上侧模板TM_A的第四代价，根据预测值6和上侧模板TM_A的重建值，确定出第一候选预测模式3关于上侧模板TM_A的第四代价。
同理，对于全部模板TM_AL，使用第一候选预测模式1、第一候选预测模式2和第一候选预测模式3分别对全部模板TM_AL进行预测，得到对应的预测值分别为预测值7、预测值8和预测值9。接着，根据预测值7和全部模板TM_AL的重建值，确定出第一候选预测模式1关于全部模板TM_AL的第四代价，根据预测值8和全部模板TM_AL的重建值，确定出第一候选预测模式2关于全部模板TM_AL的第四代价，根据预测值9和全部模板TM_AL的重建值，确定出第一候选预测模式3关于全部模板TM_AL的第四代价。
根据上述方法，确定出第j个初始预测模式对应的M个第一候选预测模式中，每一个第一候选预测模式关于P个子模板中每个子模板的第四代价，接着，执行如下S102-B21-3。
S102-B21-3、根据M个第一候选预测模式分别关于第p个子模板的第四代价，确定第j个初始预测模式关于第p个子模板的第二改善预测模式。
例如，将M个第一候选预测模式中关于第p个子模板的最小第四代价对应的第一候选预测模式，确定为第j个初始预测模式关于第p个子模板的第二改善预测模式。
继续参照上述示例，假设第j个初始预测模式对应3个第一候选预测模式，分别记为第一候选预测模式1、第一候选预测模式2和第一候选预测模式3，假设P个子模板包括左侧模板TM_L、上侧模板TM_A和全部模板TM_AL。对于左侧模板TM_L，假设第一候选预测模式1关于上侧模板TM_A的第四代价小于第一候选预测模式2关于上侧模板TM_A的第四代价，第一候选预测模式2关于上侧模板TM_A的第四代价小于第一候选预测模式3关于上侧模板TM_A的第四代价，这样可以将第一候选预测模式1确定为第j个初始预测模式关于左侧模板TM_L的第二改善预测模式。
同理，对于上侧模板TM_A，假设第一候选预测模式2关于上侧模板TM_A的第四代价小于第一候选预测模式1关于上侧模板TM_A的第四代价，第一候选预测模式1关于上侧模板TM_A的第四代价小于第一候选预测模式3关于上侧模板TM_A的第四代价，这样可以将第一候选预测模式2确定为第j个初始预测模式关于上侧模板TM_A的第二改善预测模式。
同理，对于全部模板TM_AL，假设第一候选预测模式2关于全部模板TM_AL的第四代价小于第一候选预测模式1关于全部模板TM_AL的第四代价，第一候选预测模式1关于全部模板TM_AL的第四代价小于第一候选预测模式3关于全部模板TM_AL的第四代价，这样可以将第一候选预测模式2确定为第j个初始预测模式关于全部模板TM_AL的第二改善预测模式。
根据上述方法，可以确定出K个初始预测模式中，每个初始预测模式关于P个子模板中每个子模板的第二改善预 测模式。
示例性的，假设K＝2，即2个初始预测模式分别记为第一预测模式和第二预测模式，假设P个子模板为左侧模板TM_L、上侧模板TM_A和全部模板TM_AL，根据上述方法，确定出第一预测模式和第二预测模式中每个初始预测模式关于左侧模板TM_L、上侧模板TM_A和全部模板TM_AL中每个子模板的第二改善预测模式如表6所示：
表6
Figure PCTCN2022086453-appb-000003
S102-B22、根据第i个候选权重导出模式，从K个初始预测模式分别关于P个子模板的第二改善预测模式中，确定第i个候选权重导出模式对应的K个第二改善预测模式。
具体的，针对N个候选权重导出模式中的第i个候选权重导出模式，根据该第i个候选权重导出模式所对应的子模板，从K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式中，确定出该第i个候选权重导出模式对应的K个第二改善预测模式。
例如，以K＝2为例，第i个候选权重导出模式对应的子模板为上侧模板TM_A和全部模板TM_AL，参照上述表6，可以确定出第一预测模式关于上侧模板TM_A的第二改善预测模式为第二改善预测模式12，第二预测模式关于全部模板TM_AL的第二改善预测模式为第二改善预测模式23，进而可以确定出第i个候选权重导出模式对应的2个第二改善预测模式分别为第二改善预测模式12和第二改善预测模式23。
在一种可能的实现方式中，解码端确定第i个候选权重导出模式对应的划分角度，根据划分角度和预测模式的模板之间的对应关系(具体参照上述表5所示)，从K个初始预测模式分别关于P个子模板的第二改善预测模式中，确定出第i个候选权重导出模式对应的K个第二改善预测模式。
示例性的，以K＝2为例，根据上述表2，确定出第i个候选权重导出模式对应的角度索引，例如第i个候选权重导出模式为27，则对应的角度索引为12，在上述表5所示的划分角度和预测模式的模板之间的对应关系中，查询该角度索引下，第一预测模式对应的模板和第二预测模式对应的模板，再根据第一预测模式对应的模板和第二预测模式对应的模板在上述表6中，确定出该第i个候选权重导出模式对应的2个第二改善预测模式，例如，从表5中查询出该角度索引下，第一预测模式对应的模板为上侧模板TM_A，第二预测模式对应的模板为全部模板TM_AL，这样从表6中，第一预测模式对应的第二改善预测模式中上侧模板TM_A对应的第二改善预测模式12，以及第二预测模式对应的第二改善预测模式中全部模板TM_AL对应的第二改善预测模式23，确定为第i个候选权重导出模式对应的2个第二改善预测模式。
根据上述方法，确定出N个候选权重导出模式中，第i个候选权重模式对应的K个第二改善预测模式，接着执行如下S102-B23的步骤。
S102-B23、根据第i个候选权重导出模式，以及第i个候选权重导出模式对应的K个第二改善预测模式，确定第i个候选权重导出模式对应的第一代价。
具体的，使用第i个候选权重导出模式和K个第二改善预测模式对当前块的模板进行预测，得到模板的第三预测值，根据模板的第三预测值和模板的重建值，确定第i个权重导出模式对应的第一代价。例如，根据第i个候选权重导出模式确定模板权重，使用K个第二预测改善模式分别对当前块的模板进行预测，得到K个预测值，使用确定的模板权重对该K个预测值进行加权处理，得到第i个候选权重导出模式对应的模板预测值，进而根据模板预测值和模板的重建值，确定该第i个候选权重导出模式对应的第一代价。
上述以确定第i个候选权重导出模式对应的第一代价为例，针对N个候选权重导出模式中的每个候选权重导出模式，均可以采用上述确定第i个权重导出模式对应的第一代价的方式，确定出N个候选权重导出模式中每个候选权重导出模式对应的第一代价，进而根据N个候选权重导出模式中各候选权重导出模式对应的第一代价，确定当前块的权重导出模式。
例如，将N个候选权重导出模式中第一代价最小的候选权重导出模式，确定为当前块的权重导出模式。
再例如，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表；解码码流，得到权重导出模式索引；将候选权重导出模式列表中权重导出模式索引对应候选权重导出模式，确定为当前块的权重导出模式。
在该方式三中，使用P个子模板对K个初始预测模式进行改善，得到K个初始预测模式中，每个初始预测模式关于P个子模板中每个子模板的第二改善预测模式。这样，在后期确定N个候选权重导出模式中每个候选权重导出模式对应的第一代价时，不再根据不同的候选权重导出模式对K个初始预测模式分别进行改善，而是统一从上述确定的K个初始预测模式中，每个初始预测模式关于P个子模板中每个子模板的第二改善预测模式中，查询各候选权重导出模式对应的K个第二改善预测模式，进而降低了对初始预测模式进行改善的复杂度，且提高了计算各候选权重导出模式 对应的第一代价的速度，从而可以快速从N个候选权重导出模式中确定出当前块的权重导出模式。
在该方式三中，基于P个子模板，确定出了N个候选权重导出模式中，每个候选权重导出模式对应的K个第二改善预测模式，在此基础上，上述S103-A中根据当前块的权重导出模式，确定K个调整的预测模式的方式包括但不限于如下几种：
方式1，将当前块的权重导出模式对应的K个第二改善预测模式，确定为K个调整的预测模式。
上述方式三，在确定N个候选权重导出模式中，每个候选权重导出模式对应的第一代价时，确定出了每个候选权重导出模式对应的K个第二改善预测模式，这样根据第一代价确定出当前块的权重导出模式后，可以直接将该当前块的权重导出模式对应的K个第二改善预测模式，确定为K个调整的预测模式。
方式2，为了进一步提高K个调整的预测模式的精确度，则对当前块的权重导出模式对应的K个第二改善预测模式进行改善，以得到K个调整的预测模式，此时，上述S103-A包括如下步骤：
S103-A3、根据当前块的权重导出模式，对K个第二改善预测模式进行改善，得到K个调整的预测模式。
其中，S103-A3的实现方式包括但不限于如下几种示例：
示例1，确定与当前块的权重导出模式相关的至少一个预测模式，这至少一个预测模式可以是预测角度与当前块的权重导出模式的分界线平行或近似平行的预测模式或者为预测角度与当前块的权重导出模式的分界面垂值或近似垂直的预测模式。将K个第二改善预测模式中的至少一个第二改善预测模式，使用与当前块的权重导出模式相关的至少一个预测模式中的预测模式进行替换，形成多个新的预测模式组合，每个新的预测模式组合中包括K个预测模式，针对每个新的预测模式组合，使用该新的预测模式组合所包括的K个预测模式和当前块的权重导出模式对目标进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该新的预测模式组合对应的代价，进而可以获得上述形成的多个新的预测模式组合中每一个新的预测模式组合对应的代价。同理，使用当前块的权重导出模式和K个第二改善预测模式对当前块进行预测，确定K个第二改善预测模式对应的代价。这样，解码端可以根据新的预测模式组合对应的代价，以及K个第二改善预测模式对应的代价，得到K个调整的预测模式，例如若新的预测模式组合1对应的代价最小，则将该新的预测模式组合1所包括的K个预测模式，确定为K个调整的预测模式，该新的预测模式组合1中可能包括至少一个初始预测模式，也可能不包括初始预测模式。
示例2，通过如下S103-A31至S103-A34的步骤对K个第二预测模式进行改善，得到K个调整的预测模式：
S103-A31、根据当前块的权重导出模式，确定第一模板权重。
S103-A32、针对K个第二改善预测模式中的第k个第二改善预测模式，确定第k个第二改善预测模式对应的P个第二候选预测模式，P为大于1的正整数，k为从1到K的正整数。
需要说明的是，上述S103-A32与上述S103-A31在执行过程中没有先后顺序，即上述S103-A32可以在上述S103-A31之后执行，也可以在S103-A31之前执行，或者与上述S103-A31同步执行，本申请实施例对此不做限制。
在该示例中，解码端使用当前块的权重导出模式对K个第二改善预测模式中每个第二改善预测模式进行改善，得到K个第二改善预测模式中每个第二改善预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
其中，确定K个第二改善预测模式中每个第二改善预测模式对应的调整的预测模式的过程相同，为了便于描述，本申请实施例以确定K个第二改善预测模式中第k个第二改善预测模式对应的调整的预测模式为例进行说明。
本申请实施例中，解码端针对第k个第二改善预测模式，首先确定该第k个第二改善预测模式对应的P个第二候选预测模式。
解码端确定第k个第二改善预测模式对应的P个第二候选预测模式的方式包括如下两种情况：
情况1，若第k个第二改善预测模式为帧内预测模式，将与第k个第二改善预测模式相似的P个帧内预测模式，确定为P个第二候选预测模式。例如，第k个第二改善预测模式为角度预测模式，则将与第k个第二改善预测模式的角度相似(或相近)的P个角度预测模式，确定第k个第二改善预测模式对应的P个第二候选预测模式。
情况2，若第k个第二改善预测模式为帧间预测模式，则根据第k个第二改善预测模式进行搜索，得到P个第二运动信息，根据这P个第二运动信息，确定第k个第二改善预测模式对应的P个第二候选预测模式。例如，对于第k个第二改善预测模式，其运动矢量是(xInit，yInit)，设置一个搜索范围如水平方向从xInit-sR到xInit+sR，竖直方向从yInit-sR到yInit+sR的矩形区域，其中sR可以是2，4，8等。该矩形区域内的每一个运动矢量都可以和第k个第二改善预测模式的其他信息，如参考帧索引和预测列表标志等组合起来确定一个运动信息，从而确定一个预测模式，这样可以获得第k个第二改善预测模式对应的P个第二候选预测模式。
在一些实施例中，上述根据第k个第二改善预测模式进行搜索，得到P个第二运动信息的一种方式为，根据第k个第二改善预测模式对应的运动矢量，在预设搜索范围内进行搜索，得到P个偏移量，每个偏移量包括第一方向(例如x方向)上的偏移量和第二方向(例如y方向)上的偏移量。进而，根据这M个偏移量，得到P个第二运动信息，例如，对于P个偏移量中的每个偏移量，根据该偏移量对第k个第二改善预测模式对应的运动矢量进行偏移，得到一个第二运动信息。
S103-A33、根据第一模板权重，确定使用P个第二候选预测模式分别对模板进行预测时所对应的第五代价。
根据上述情况1或情况2的方法，确定出第k个第二改善预测模式对应的P个第二候选预测模式后，确定这P个第二候选预测模式对应的代价，进而根据代价确定第k个第二改善预测模式对应的调整的预测模式。为了便于描述，将第二候选预测模式对应的代价，记为第五代价。
具体的，针对P个第二候选预测模式中的每个第二候选预测模式，使用该第二候选预测模式对当前块的模板进行 预测，得到一个预测值，接着，根据该第一模板权重和该预测值确定该第一候选预测模式关于模板的第五代价。以模板上的一个像素点为例，使用该第二候选预测模式对该像素点进行预测，得到该像素点的一个预测值，在第一模板权重中确定该像素点对应的模板权重，将该像素点的预测值与模板权重的乘积，确定该第二候选预测模式在该像素点处的预测值。参照该方式，可以得到该第二候选预测模式在模板中每个像素点处的预测值，这些预测值组成该第二候选预测模式对应的模板预测值，根据模板预测值和模板的重建值，确定出该第二候选预测模式对应的第五代价。
根据上述方法，可以确定出P个第二候选预测模式中每个第二候选预测模式对应的第五代价，接着，执行如下S103-A14。
S103-A34、根据P个第二候选预测模式分别对应的第五代价，确定第k个第二改善预测模式对应的调整的预测模式。
例如，将P个第二候选预测模式中第五代价最小的第二候选预测模式，确定为第k个第二改善预测模式对应的调整的预测模式。
上述实施例以确定K个第二改善预测模式中的第k个第二改善预测模式对应的调整的预测模式为例进行说明，参照上述描述，可以确定出K个第二改善预测模式中每个第二改善预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
在该方式2中，解码端首先使用P个子模板对K个初始预测模式进行改善，得到当前块的权重导出模式对应的K个第二改善预测模式，再使用当前块的权重导出模式对当前块的权重导出模式对应的K个第二改善预测模式进行改善，得到精准的K个调整的预测模式，这样使用这K个调整的预测模式和当前块的权重导出模式对当前块进行预测时，可以进一步提高预测的准确性。
本申请实施例，通过上述方法，可以确定出当前块的权重导出模式和K个调整的预测模式，进而使用当前块的权重导出模式和K个调整的预测模式，对当前块进行预测，得到当前块的预测值。
本申请中根据当前块的权重导出模式和K个调整的预测模式，确定当前块的预测值的方式包括但不限于如下几种：
方式一，若在上述根据当前块的权重导出模式确定第一模板权重时，通过当前块的权重导出模式确定出第一模板权重，但未确定出当前块中像素点对应的预测值的权重，在该情况下，则上述S103-B包括如下步骤：
S103-B11、根据K个调整的预测模式，确定K个预测值；
S103-B12、根据当前块的权重导出模式，确定预测值的权重；
S103-B13、根据K个预测值和该预测值的权重，确定当前块的预测值。
需要说明的是，在本申请中，当前块的权重导出模式用于对当前块使用的预测值的权重进行确定。具体地，当前块的权重导出模式可以是导出预测值的权重的模式。对于一个给定长度和宽度的块，每一种权重导出模式可以导出一个预测值的权重矩阵；对于同样大小的块，不同权重导出模式导出的预测值的权重矩阵可以不同。
可以理解的是，在本申请的实施例中，解码端在基于K个调整的预测模式以及预测值的权重，确定当前块的预测值时，可以先根据K个调整的预测模式确定K个预测值；然后利用预测值的权重对K个预测值进行加权平均计算，最终便可以获得当前块的预测值。
在该方式一中，可以理解根据当前块的权重导出模式确定第一模板权重和预测值的权重是两个相互独立的过程，互不干涉。
在一些实施例中，本申请还可以采用下面方式二，确定出当前块的预测值。
方式二，若在上述确定第一模板权重时，通过当前块的权重导出模式确定出模板区域和当前块构成的合并区域中像素点的权重，此时，S103-B包括如下步骤：
S103-B21、根据K个调整的预测模式，确定K个预测值；
S103-B22、根据合并区域中像素点的权重，确定K个预测值的权重；
S103-B23、根据K个预测值和K个预测值的权重，确定当前块的预测值。
在该方式二中，在权重导出时，根据当前块的权重导出模式导出合并区域中像素点的权重，而该合并区域包括当前块和当前块的模板区域，进而将合并区域中当前块对应的权重确定为预测值的权重，将合并区域中模板区域对应的权重确定为第一模板权重。也就是说，在该方式二中，将模板区域和当前块作为一个整体区域，可以一步导出第一模板权重和预测值的权重，减少了权重导出的步骤，提高预测效果。
在一些实施例中，上述预测过程是以像素点为单位进行的，对应的上述权重也为像素点对应的权重。此时，对当前块进行预测时，使用K个调整的预测模式中的每个调整的预测模式对当前块中的某一个像素点A进行预测，得到K个调整的预测模式关于像素点A的K个预测值，根据当前块的权重导出模式确定像素点A的K个预测值的权重，对这K个预测值进行加权，得到像素点A的预测值。对当前块中的每一个像素点执行上述步骤，可以得到当前块中每个像素点的预测值，当前块中每个像素点的预测值构成当前块的预测值。以K＝2为例，使用第一预测模式对当前块中的某一个像素点A进行预测，得到该像素点A的第一预测值，使用第二预测模式对该像素点A进行预测，得到该像素点A的第二预测值，根据像素点A对应的预测值权重，对第一预测值和第二预测值进行加权，得到像素点A的预测值。
在一种示例中，以K＝2为例，若第一预测模式和第二预测模式均为帧内预测模式时，采用第一帧内预测模式进行预测，得到第一预测值，采用第二帧内预测模式进行预测，得到第二预测值，根据预测值的权重对第一预测值和第二预测值进行加权，得到当前块的预测值。例如，采用第一帧内预测模式对像素点A进行预测，得到像素点A的第一预测值，采用第二帧内预测模式对像素点A进行预测，得到像素点A的第二预测值，根据像素点A对应的预测值的权 重，对第一预测值和第二预测值进行加权，得到像素点A的预测值。
在一些实施例中，若K个调整的预测模式中的第j个调整的预测模式为帧间预测模式，则上述S103-B根据K个调整的预测模式以及当前块的权重导出模式，确定当前块的预测值包括如下步骤：
S103-B31、根据第j个调整的预测模式，确定运动信息；
S103-B32、根据运动信息，确定第j个预测值；
S103-B33、根据K个调整的预测模式中除第j个调整的预测模式外的其他预测模式，确定K-1个预测值；
S103-B34、根据当前块的权重导出模式，确定K个预测值的权重；
S103-B35、根据第j个预测值、K-1个预测值和预测值的权重，确定当前块的预测值。
以K＝2为例，若第一预测模式为帧内预测模式，第二预测模式为帧间预测模式时，采用帧内预测模式进行预测，得到第一预测值，采用帧间预测模式进行预测，得到第二预测值，根据预测值的权重对第一预测值和第二预测值进行加权，得到当前块的预测值。在该示例中，采用帧内预测模式对当前块中每一个点进行预测，得到当前块中每一个点的预测值，当前块中每一个点的预测值，构成当前块的第一预测值。采用帧间预测模式，确定一运动信息，根据该运动信息确定当前块的最佳匹配块，将该最佳匹配块确定为当前块的第二预测值。针对当前块中每个像素点的预测值的权重，对当前块的第一预测值和第二预测值进行逐点加权运算，得到当前块的预测值。例如，对于当前块中的像素点A，根据像素点A的预测值的权重，将当前块的第一预测值中像素点A对应的第一预测值，与当前块的第二预测值中像素点A对应的第二预测值进行加权，得到像素点A的预测值。
在一些实施例中，若K大于2时，则可以根据当前块的权重导出模式确定K个调整的预测模式中任意两个调整的预测模式对应的预测值的权重，K个调整的预测模式中的其他调整的预测模式对应的预测值的权重可以为预设值。例如，K＝3，第一预测模式和第二预测模式对应的预测值的权重根据当前块的权重导出模式导出，第三预测模式对应的预测值的权重为预设值。在一些实施例中，若K个调整的预测模式对应的预测值的总权重一定，例如为8，则可以根据预设权重比例，来确定K个调整的预测模式各自对应的预测值的权重，假设第三预测模式对应的预测值的权重占整个中预测值的总权重的1/4，则可以确定第三预测模式的预测值的总权重为2，预测值的总权重中的剩下3/4分配给第一预测模式和第二预测模式。示例性的，如果根据当前块的权重导出模式导出第一预测模式对应的预测值的权重3，则确定第一预测模式对应的预测值的权重为(3/4)*3，第二预测模式对应的预测值的权重为第一预测模式的预测值的权重为(3/4)*5。
解码端根据上述方法，确定出当前块的预测值，另外，解码端通过解码码流，得到当前块的量化系数，对量化系数进行反量化，得到变换系数，对变换系数进行反变换，得到当前块的残差值。当前块的预测值和残差值进行相加，得到当前块的重建值。
本申请实施例提供的预测方法，解码端解码码流，确定K个初始预测模式；根据所述K个初始预测模式，确定当前块的权重导出模式；根据当前块的权重导出模式，确定当前块的预测值。即本申请通过K个初始预测模式，确定当前块的权重导出模式，进而丰富了权重导出模式的确定方式。另外，本申请中编码端在码流中可以不传输权重导出模式的索引，解码端也可以根据K个初始预测模式确定出当前块的权重导出模式，进而节约了码字，降低了编码代价。
上文以解码端为例对本申请的预测方法进行介绍，下面以编码端为例进行说明。
图19为本申请实一施例提供的预测方法流程示意图，本申请实施例应用于图1和图2所示视频编码器。如图19所示，本申请实施例的方法包括：
S201、确定K个初始预测模式。
需要说明的是，在本申请的实施例中，视频图像可以划分为多个图像块，当前块即为每个当前待编码的图像块，可以称为编码块(Coding Block，CB)。这里，每个编码块可以包括第一图像分量、第二图像分量和第三图像分量。具体地，在本申请中，假定进行第一预测，而且第一图像分量为亮度分量，即待预测图像分量为亮度分量，那么可以将待预测的编码块称为亮度块；或者，假定进行第二预测，而且第二图像分量为色度分量，即待预测图像分量为色度分量，那么可以将待预测的编码块称为色度块。
由上述可知，为了进一步提高预测准确性，可以使用多个预测模式对当前块进行预测，此时，为了保持编码端和解码端的预测一致性，在一些实施例中，编码端需要在码流中需要指示权重导出模式，例如在码流中写入权重导出模式的索引，以VVC为例，GPM的64种权重导出模式的编码是按等概率处理的，即64种权重导出模式在进行二值化和反二值化的时候都使用等长的6比特，这样则增加了编码代价。
本申请实施例，为了降低编码代价，解码端通过预测模式导出权重导出模式。具体是，解码端通过解码码流，确定K个初始预测模式，接着，根据这K个初始预测模式，确定当前块的权重导出模式，进而根据当前块的权重导出模式，确定当前块的预测值。对应的，编码端也采用与解码端相同的预测方式，即确定K个初始预测模式，并根据K个初始预测模式，确定当前块的权重导出模式，再根据当前块的权重导出模式，确定当前块的预测值。
在一些实施例中，为了便于描述，将当前块的权重导出模式记为当前块的权重导出模式。
上述当前块的K个初始预测模式包括如下几种示例：
示例1，上述K个初始预测模式均为帧内预测模式。
示例2，上述K个初始预测模式均为帧间预测模式。
示例3，上述K个初始预测模式中，至少一个为帧内预测模式，至少一个为帧间预测模式。
示例4，上述K个初始预测模式中，至少一个为帧内预测模式，至少一个为非帧间和非帧内的预测模式，例如为块内复制IBC预测模式或调色板palette预测模式等。
示例5，上述K个初始预测模式中，至少一个为帧间预测模式，至少一个为非帧间和非帧内的预测模式，例如为IBC预测模式或palette预测模式等。
示例6，上述K个初始预测模式均不是帧内预测模式，也不是帧间预测模式，例如一个为IBC预测模式，一个为 palette预测模式等。
需要说明的是，本申请实施例对上述K个初始预测模式的具体类型不做限制。
其中，上述S201中确定K个初始预测模式的方式包括但不限于如下几种：
方式1，上述K个初始预测模式为预设的K个预测模式。
方式2上述上述S201包括S201-A1至S201-A2的步骤：
S201-A1、确定备选预测模式列表，该备选预测模式列表中包括至少两个备选预测模式；
S201-A3、根据备选预测模式列表，确定出K个初始预测模式。
在一些实施例中，若上述K个初始预测模式均为帧内预测模式，则该备选预测模式列表所包括的备选预测模式均为帧内预测模式，例如包括直流(Direct Current，DC)模式、平面(PLANAR)模式、角度模式等中的至少一个。可选的，上述备选预测模式列表还可以包括MPM列表中的帧内预测模式。
在一些实施例中，若上述K个初始预测模式均为帧间预测模式，则该备选预测模式列表所包括的备选预测模式均为帧间预测模式，例如包括skip、merge、普通帧间预测模式等中的至少一个。
在一些实施例中，若上述K个初始预测模式中即包括帧间预测模式又包括帧内预测模式，则该备选预测模式列表包括至少一个帧内预测模式和至少一个帧间预测模式。
可选的，上述备选预测模式列表还可以包括IBC、palette等模式。
可选的，上述备选预测模式列表还可以包括单向预测、双向预测、多假设预测等。
本申请对上述备选预测模式列表所包括的备选预测模式的类型和个数不做限制。
本申请实施例对确定备选预测模式列表的方式不做限制。
在一种示例中，上述备选预测模式列表所包括的备选预测模式为预设模式。
在一种示例中，上述备选预测模式列表为MPM列表。
在一种示例中，上述备选预测模式列表是根据一些规则，如等间距筛选等，确定出的预测模式的集合。
在一种示例中，编码端和解码端均默认上述备选预测模式列表中的备选预测模式是通过模板匹配方式确定的，例如，编码端使用预测模式对当前块的模板进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该预测模式的代价，例如SAD代价、SATD代价或MSE代价等。这样编码端可以根据各预测模式的代价对预测模式进行排序，得到备选预测模式列表，例如将代价最小的多个预测模式确定为备选预测模式，构成备选预测模式列表。
本申请实施例中，根据上述方法，确定出备选预测模式列表后，从备选预测模式列表中选出K个备选预测模式，作为K个初始预测模式。
示例性的，编码端从备选预测模式列表中随机选出K个备选预测模式，作为K个初始预测模式。
示例性的，编码端将备选预测模式列表中代价最小的K个备选预测模式，确定为K个初始预测模式。
编码端确定出K个初始预测模式后，在码流中写入预测模式索引，该预测模式索引可以是K个初始预测模式中至少一个初始预测模式的索引，例如该预测模式索引为K个初始预测模式的索引，或者K个初始预测模式中第一个初始预测模式的索引。这样解码端解码码流，得到该预测模式索引，并根据该预测模式索引从确定的备选预测模式列表中，确定出K个初始预测模式。
本申请实施例中，编码端根据上述步骤，确定出K个初始预测模式，根据这K个初始预测模式确定当前块的权重导出模式，再根据该当前块的权重导出模式对当前块进行预测，确定当前块的预测值。
在一些实施例中，编码端在确定K个初始预测模式之前，首先需要判断当前块是否使用K种不同的预测模式进行加权预测处理。若编码端确定当前块使用K种不同的预测模式进行加权预测处理时，则执行上述S201确定K个初始预测模式。
在一种可能的实现方式中，编码端可以通过确定当前块的预测模式参数，来确定当前块是否使用两种不同的预测模式进行加权预测处理。
可选的，在本申请的实施中，预测模式参数可以指示当前块是否可以使用GPM模式或AWP模式，即指示当前块是否可以使用K种不同的预测模式进行预测处理。
可以理解的是，在本申请的实施例中，可以将预测模式参数理解为一个表明是否使用了GPM模式或AWP模式标志位。具体地，编码端可以使用一个变量作为预测模式参数，从而可以通过对该变量的取值的设置来实现预测模式参数的设置。示例性的，在本申请中，如果当前块使用GPM模式或AWP模式，那么编码端可以将预测模式参数的取值设置为指示当前块使用GPM模式或AWP模式，具体地，编码端可以将变量的取值设置为1。示例性的，在本申请中，如果当前块不使用GPM模式或AWP模式，那么编码端可以将预测模式参数的取值设置为指示当前块不使用GPM模式或AWP模式，具体地，编码端可以将变量取值设置为0。进一步地，在本申请的实施例中，编码端在完成对预测模式参数的设置之后，便可以将预测模式参数写入码流中，传输至解码端，从而可以使解码端在解析码流之后获得预测模式参数。
基于此，编码端根据该预测模式参数确定当前块是否使用GPM模式或AWP模式，若当前块使用GPM模式或AWP模式，即使用K个不同的预测模式进行预测处理时，确定K个初始预测模式。
需要说明的是，在本申请的实施例中，GPM模式或AWP模式为一种预测方法，具体地，为当前块确定K个不同的预测模式，然后分别根据这K个不同的预测模式确定出K个预测值，接着可以再确定权重，将则会K个预测值依据权重进行组合，最终便可以得到当前块的预测值。
在一些实施例中，在应用GPM模式或AWP模式时，可以对当前块的尺寸进行限制。可以理解的是，由于本申请实施例提出的预测方法需要分别使用K个不同的预测模式生成K个预测值，再根据权重进行加权得到当前块的预测 值，为了降低的复杂度，同时考虑压缩性能和复杂度的权衡，在本申请的实施例中，可以限制对一些大小的块不使用该GPM模式或AWP模式。因此，在本申请中，编码端可以先确定当前块的尺寸参数，然后根据尺寸参数确定当前块是否使用GPM模式或AWP模式。
需要说明的是，在本申请的实施例中，当前块的尺寸参数可以包括当前块的高度和宽度，因此，编码端可以利用当前块的高度和宽度对使用GPM模式或AWP模式进行限制。示例性的，在本申请中，若宽度大于第一阈值且高度大于第二阈值，则确定当前块使用GPM模式或AWP模式。可见，一种可能的限制是仅仅在块的宽度大于(或大于等于)第一阈值，且块的高度大于(或大于等于)第二阈值的情况下使用GPM模式或AWP模式。其中，第一阈值和第二阈值的值可以是8，16，32等，第一阈值可以等于第二阈值。示例性的，在本申请中，若宽度小于第三阈值且高度大于第四阈值，则确定当前块使用GPM模式或AWP模式。可见，一种可能的限制是仅仅在块的宽度小于(或小于等于)第三阈值，且块的高度大于(或大于等于)第四阈值的情况下使用AWP模式。其中，第三阈值和第四阈值的值可以是8，16，32等，第三阈值可以等于第四阈值。
在一些实施例中，在本申请的实施例中，还可以通过像素参数的限制来实现限制能够使用GPM模式或AWP模式的块的尺寸。示例性的，在本申请中，编码端可以先确定当前块的像素参数，然后再根据像素参数和第五阈值进一步判断当前块是否可以使用GPM模式或AWP模式。可见，一种可能的限制是仅仅在块的像素数大于(或大于等于)第五阈值的情况下使用GPM模式或AWP模式。其中，第五阈值的值可以是8，16，32等。也就是说，在本申请中，只有在当前块的尺寸参数满足大小要求的条件下，当前块才可以使用GPM模式或AWP模式。
在一些实施例中，在本申请中，可以有一个帧级的标志来确定当前待解码帧是否使用本申请的技术方案。如可以配置帧内帧(如I帧)使用本申请，帧间帧(如B帧、P帧)不使用本申请。或者可以配置帧内帧不使用本申请，帧间帧使用本申请。或者可以配置某些帧间帧使用本申请，某些帧间帧不使用本申请。帧间帧也可以使用帧内预测，因而帧间帧也有可能使用本申请。
在一些实施例中，还可以有一个帧级以下、CU级以上(如tile、slice、patch、LCU等)的标志来确定这一区域是否使用本申请的技术方案。
S202、根据K个初始预测模式，确定当前块的权重导出模式。
在本申请中，权重导出模式用于对当前块使用的权重进行确定。具体地，权重导出模式可以是导出权重的模式。对于一个给定长度和宽度的块，每一种权重导出模式可以导出一个权重矩阵；对于同样大小的块，不同权重导出模式导出的权重矩阵不同。
示例性的，在本申请中，AWP有56种权重导出模式，GPM有64种权重导出模式。
需要说明的是，为了便于描述，本申请实施例将最终确定的用于当前块预测的权重导出模式，记为当前块的权重导出模式。
本申请实施例中，编码端根据上述确定的K个初始预测模式，确定当前块的权重导出模式，在一些情况下，编码端可以在码流中不写入当前块的权重导出模式相关的信息，进而节省码字，降低编码代价。
本申请实施例对编码端根据K个初始预测模式，确定当前块的权重导出模式的方式不做限制。
在一些实施例中，编码端基于当前块的模板，根据K个初始预测模式，确定当前块的权重导出模式。以VVC为例，GPM有64种权重导出模式，编码端通过对模板进行分析来估计哪些权重导出模式被选中的可能性大，哪些权重导出模式被选中的可能性小。以帧间的GPM为例，一个典型的场景是2个物体相互运动，GPM可以用于2个物体的边缘。假设当前块与模板中都包含物体的边缘，那么就可以用模板上的边缘来推测当前块的边缘。一个例子如下，给定2个运动信息，根据这2个运动信息分别得到2个模板上的预测值，记为第一模板预测值和第二模板预测值，根据某一权重导出模式导出模板权重，根据第一模板预测值，第二模板预测值和模板权重，确定该权重导出模式对应的模板预测值，因为模板的重建值是可得的，所以根据该权重导出模式对应的模板预测值和模板的重建值可以得到该权重导出模式在模板上的代价。这样可以根据各个权重导出模式在模板上的代价估计它被选中的概率。一种是根据各个权重导出模式在模板上的代价对权重导出模式进行排序。代价小的模式在前，代价大的模式在后。因为认为经过排序的权重导出模式大致是从概率大到概率小排列的，所以可以使用变长编码的方法来处理。一种可能的实现方式是，构建一个权重导出模式的列表，编码端把当前块选择的当前块的权重导出模式按列表中的索引写入码流中，解码器解析码流中的索引，从列表中确定当前块的权重导出模式。
由上述可知，本申请实施例在根据K个初始预测模式，确定当前块的权重导出模式过程中，需要根据权重导出模式，确定模板的权重。
下面对本申请实施例涉及的根据权重导出模式，确定模板权重的过程进行介绍。
模板匹配是利用相邻像素之间的相关性，把当前块周边的一些区域作为模板。在当前块进行编解码时，按照编码顺序其左侧及上侧已经解码完成。在帧间预测时，在参考帧中寻找模板的最佳匹配位置从而确定当前块的运动信息或者说运动矢量。在帧内预测时，利用模板来确定当前块的帧内预测模式。
本申请对当前块的模板的具体形状不做限制。
在一些实施例中，模板包括当前块的上方已解码区域、左侧已解码区域和左上方已解码区域中的至少一个。
可选的，上方已解码区域的宽度与当前块的宽度相同，左侧已解码区域的高度与当前块的高度相同，左上方已解码区域的宽度与左侧已解码区域的宽度相同，左上方已解码区域的高度与上方已解码区域的高度相同。
目前，如上述表5所示，以K＝2为例，K个预测模式包括第一预测模式和第二预测模式，第一预测模式和第二预测模式对应的模板为当前块的上方已解码区域，或者为当前块左侧已解码区域，或者为当前块的左侧已解码和上方已解码区域。例如图16所示，以GPM的索引为2的权重导出模式为例，当前块的权重矩阵的白色区域为第一预测模式的预测值对应的权重，黑色区域为第二预测模式的预测值对应的权重。如图16所示，第一预测模式对应的第一模板为当前块的上方已解码区域，第二预测模式对应的第二模板为当前块的左侧已解码区域，但是，与第二预测模式靠近的 模板包括左侧区域外，还包括部分上方已解码区域，因此，现有技术对于模板的划分不够精细，进而无法基于模板准确出当前块的权重导出模式。
为了解决该技术问题，本申请可以通过权重导出模式实现模板的更精细划分，例如图17A所示，本申请将当前块的权重导出模式对应的权重矩阵的分割线向模板区域延伸，以将模板进行划分，例如将分割线一侧的模板记为第一模板，将分割线另一侧侧的模板记为第二模板。第一模板对应第一预测模式，第二模板对应的第二预测模式，第一模板的权重根据第一预测模式对应的权重确定，第二模板的权重根据第二预测模式对应的权重确定，例如第一模板的权重与第一预测模式对应的权重相同，第二模板的权重与第二预测模式对应的权重相同。
在一些实施例中，根据上述方法划分的第一模板和第二模板可能不是长方形，例如图17A所示，第一模板和第二模板具有斜边，对不规则的模板计算代价较复杂。
为了降低模板匹配的复杂度，在一些实施例中，可以直接将权重矩阵向模板区域延伸，例如向左向上延伸，将权重矩阵覆盖到模板上面，从而确定出模板的权重来。例如图17B所示，可以选择把当前块左上侧的区域的小矩形区域加入到模板中，将模板和当前块拼起来构成一个矩形。当然也可以只使用左侧部分和上方部分作为模板。如图17B所示，这里以加入左上侧的区域举例，左侧、左上侧和上侧的倒L形框中的区域为模板区域，右下方的矩形区域为当前块，那么向左上方扩展出来的权重矩阵就变成了模板的权重矩阵。这样不需要真正地划分模板，而是利用权重模拟划分模板。可以说是利用权重导出模式模拟划分模板，或者说利用权重导出模式确定模板权重，根据模板权重确定模板匹配时的代价。由于不同的块形状下模板的划分可能不同，而本申请则不需要对各种形状的块设置各种规则，例如不需要设置表5所示的预测模式与模板的对应关系。而就“划分”的准确性来说，本申请的模拟划分也比只区分左侧区域和上方区域的方法更精细和准确。
在一些实施例中，可以将图17B所示的当前块的左侧区域和上方区域作为当前块的模板，而不考虑左上方的矩阵区域。
在模板匹配时，可以只使用一个分量(Y、Cb、Cr或R、G、B)，比如在YUV格式中可以使用Y，因为Y占主导地位，当然也可以所有分量都使用。为了便于描述，本申请以一个分量为例进行说明，例如以Y分量进行模板匹配为例，其他分量参照即可。
本申请实施例中，根据权重导出模式导出模板权重可以与预测值的权重的导出过程合并，例如同时导出模板权重和预测值的权重。其中预测值的权重可以理解为预测值对应的权重，例如使用第一预测模式得到第一预测值，使用第二预测模式得到第二预测值，根据权重导出模式确定第一预测值的第一权重，根据权重导出模式确定第二预测值的第二权重，将第一预测值与第一权重的乘积，以及第二预测值与第二权重的乘积的和值，确定为当前块的预测值。
本申请为了与模板权重区分，将上的第一权重和第二权重称为预测值的权重。
在一些实施例中，上述根据权重导出模式，确定模板权重包括如下步骤：
步骤A、根据权重导出模式，确定角度索引和距离索引；
步骤B、根据角度索引、距离索引和模板的大小，确定模板权重。
本申请可以采用与导出预测值的权重相同的方式，导出模板权重。例如，首先根据权重导出模式，确定角度索引和距离索引，其中角度索引可以理解为权重导出模式导出的各权重的分界线角度索引。示例性的，可以根据上述表2，确定出权重导出模式对应的角度索引和距离索引，例如权重导出模式为27，则对应的角度索引为12，距离索引为3。接着，根据角度索引、距离索引和模板的大小，确定出模板权重。
上述步骤B中根据权重导出模式导出模板权重的方式包括但不限于如下几种方式：
方式一，根据角度索引、距离索引和模板的大小，直接确定模板权重。此时，上述步骤B包括如下步骤B11至步骤B13的步骤：
步骤B11、根据角度索引、距离索引和模板的大小，确定模板中像素点的第一参数；
步骤B12、根据模板中像素点的第一参数，确定模板中像素点的权重；
步骤B13、根据模板中像素点的权重，确定模板权重。
本实现方式中，根据角度索引、距离索引、模板的大小和当前块的大小，确定模板中像素点的权重，进而将模板中每个点的权重组成的权重矩阵，确定为模板权重。
本申请的第一参数用于确定权重。在一些实施例中，第一参数也称为权重索引。
在一种可能的实现方式中，可以根据上述实施例的方式确定出偏移量和第一参数，具体参照上述实施例的描述，在此不再赘述。
根据上述方法，确定出第一参数weightIdx后，根据weightIdx确定出模板中点(x，y)处的权重。
本申请中，上述步骤B12中根据模板中像素点的第一参数，确定模板中像素点的权重的方式包括的不限于如下几种：
方式1，根据模板中像素点的第一参数，确定模板中像素点的第二参数；根据模板中像素点的第二参数，确定模板中像素点的权重。
其中第二参数也用于确定权重。在一些实施例中，上述第二参数也称为第一分量下的权重索引，该第一分量可以为亮度分量、色度分量等。
方式2，根据模板中像素点的第一参数、第一预设值和第二预设值，确定模板中像素点的权重。
为了降低模板权重的计算复杂度，在方式2中将模板中像素点的权重限定为第一预设值或第二预设值，也就是说，模板中像素点的权重要么为第一预设值，要么是第二预设值，进而降低模板权重的计算复杂度。
本申请对第一预设值和第二预设值的具体取值不做限制。
可选的，第一预设值为1。
可选的，第二预设值为0。
在上述方式一中，通过权重导出模式确定出模板中每个点的权重，模板中每个点的权重组成的权重矩阵作为模板 权重。
方式二，根据权重导出模式确定出当前块和模板的权重，也就是说，在该方式二中，将当前块和模板组成的合并区域作为一个整体，根据权重导出模式导出合并区域中像素点的权重。基于此上述步骤B包括如下步骤B21和S202-B22步骤：
步骤B21、根据角度索引、距离索引、模板的大小和当前块的大小，确定当前块和模板组成的合并区域中像素点的权重；
步骤B22、根据模板的大小和合并区域中像素点的权重，确定模板权重。
在该方式二中，将当前块和模板作为一个整体，根据角度索引、距离索引、模板的大小和当前块的大小，确定当前块和模板组成的合并区域中像素点的权重，进而根据模板的大小，将合并区域中像素点的权重中模板对应的权重确定为模板权重，例如图18所示，将合并区域中L型模板区域对应的权重确定为模板权重。
该方式二中，在一次权重确定过程中，不仅确定出模板权重，并且确定出预测值的权重，即合并区域对应的权重中除去模板权重后，剩余的为预测值的权重，进而可以根据预测值的权重执行后续的预测过程，无需重新确定预测值的权重，进而减少了预测步骤，提升了预测效率。
本申请对上述根据角度索引、距离索引、模板的大小和当前块的大小，确定当前块和模板组成的合并区域中像素点的权重的具体实现方式不做限制。
在一些实施例中，上述步骤B21中确定合并区域中像素点的权重包括如下步骤B211至步骤B212的步骤：
步骤B211、根据角度索引、距离索引和合并区域的大小，确定合并区域中像素点的第一参数；
步骤B212、根据合并区域中像素点的第一参数，确定合并区域中像素点的权重。
本实现方式中，根据角度索引、距离索引、合并区域的大小，确定合并区域中像素点的权重，合并区域中每个点的权重组成权重矩阵。
具体的参照上述实施例的描述，在此不再赘述。
本申请中，上述步骤B212中根据合并区域中像素点的第一参数，确定合并区域中像素点的权重的方式包括的不限于如下几种：
方式1，根据合并区域中像素点的第一参数，确定合并区域中像素点的第二参数；根据合并区域中像素点的第二参数，确定合并区域中像素点的权重。
方式2，根据合并区域中像素点的第一参数、第一预设值和第二预设值，确定合并区域中像素点的权重。
示例性的，合并区域中像素点的权重为第一预设值或为第二预设值。
为了降低合并区域权重的计算复杂度，在方式2中将合并区域中像素点的权重限定为第一预设值或第二预设值，也就是说，合并区域中像素点的权重要么为第一预设值，要么是第二预设值，进而降低合并区域权重的计算复杂度。
本申请对第一预设值和第二预设值的具体取值不做限制。
可选的，第一预设值为1。
可选的，第二预设值为0。
在上述方式二中，通过权重导出模式确定出当前块和模板组成的合并区域中每个点的权重，进而根据模板的大小，将合并区域中模板对应的权重确定为模板权重，且该方式二中，同时确定出预测值的权重，无需执行后续的预测值的权重的确定步骤，进行降低了预测步骤，提升了预测效率。
上述对根据权重导出模式，确定模板权重的过程进行详细介绍，其中上述权重导出模式可以为本申请实施例涉及的任意一个权重导出模式，例如为当前块的权重导出模式，或者为N个候选预测模式中的第i个候选预测模式等等。
S203、根据当前块的权重导出模式，确定当前块的预测值。
本申请中，S203中编码端根据当前块的权重导出模式，确定当前块的预测值的方式包括但不限于如下几种：
方式一，编码端根据当前块的权重导出模式和K个初始预测模式，确定当前块的预测值，具体的，编码端根据当前块的权重导出模式确定预测值的权重，再根据K个初始预测模式对当前块进行预测，得到K个预测值，然后利用预测值的权重对K个预测值进行加权平均计算，得到当前块的预测值。
需要说明的是，上述根据当前块的权重导出模式导出的预测值的权重可以理解为导出当前块中每一个像素点对应的权重，也可以理解为导出当前块对应的权重矩阵。其中，基于预测值的权重确定当前块的预测值时，可以是确定当前块中每一个像素点对应的K个预测值，并根据每一个像素点对应的K个预测值和预测值的权重，确定出每个像素点对应的预测值，当前块中每一个像素点对应的预测值构成当前块的预测值。可选的，基于预测值的权重确定当前块的预测值时，还可以是按照块来执行，例如，确定当前块的K个预测值，根据当前块的权重矩阵对当前块的K个预测值进行加权，得到当前块的预测值。
方式二，编码端可以使用当前块的权重导出模式对上述K个初始预测模式进行改善，此时，上述S203包括如下S203-A和S203-B的步骤：
S203-A、根据当前块的权重导出模式，确定K个调整的预测模式；
S203-B、根据K个调整的预测模式和当前块的权重导出模式，确定当前块的预测值。
本申请实施例中，可以认为GPM有K+1个要素，即一个权重导出模式和K个预测模式。由于一个权重导出模式和K个预测模式共同产生一个预测块，这个预测块作用于当前块。它们之间是有关联的。举几个例子，比如当前块包含2个相对运动的物体的边缘，这是帧间GPM的一个理想的场景。那么理论上这个“划分”应该发生在物体的边缘，但是实际上“划分”有有限种可能，不可能覆盖任意的边缘，有时会选择相近的“划分”，那这样可能相近的“划分”不止一种，选择哪种就取决于哪种“划分”与2个预测模式组合的结果最优。同理，选择哪种预测模式有时同样取决于哪种组合的结果最优，因为即使是在使用该预测模式的部分，对自然视频而言，这部分也很难和当前块完全匹配， 最终选择的可能是编码效率最高的。另一种GPM用得比较多的地方是当前块包含一个物体中存在相对运动的部分。比如说胳膊的摆动等导致扭曲、变形的地方，这样的“划分”就更模糊，可能最终取决于哪种组合的结果最优。还有一种场景是帧内的预测，自然图像中某些部分的纹理是很复杂的，某些部分存在一种纹理向另一种纹理的渐变，某些部分可能无法用简单的一个方向来表述，帧内GPM可以提供出更复杂的预测块，而帧内编码的块相对于相同量化下的帧间编码的块通常存在更大残差，选择哪种预测模式可能最终取决于哪种组合的结果最优。
可见K+1个要素之间是会互相影响的，因而可以利用权重导出模式确定预测模式，也可以利用预测模式确定权重导出模式。更进一步地，可以看到某些模式信息也是可以进行改善(refine)的。所谓改善，就是在一个初始的模式信息的基础上，通过采集更多的相关信息，使用更多的计算等手段，获取一个更好的模式信息。一般说的refine是编码端的操作。而基于模板也可以对运动信息进行改善refine。即在初始的运动信息的基础上，在编码端进行一个小范围的搜索，根据所搜索的运动信息与模板的匹配程度来优化运动信息。所谓运动信息与模板的匹配程度一种理解是运动信息在模板上的预测值与模板的重建值的匹配程度。
对应到GPM，可以先根据预测模式确定权重导出模式，再返回来根据权重导出模式refine预测模式。或者先根据权重导出模式确定预测模式，再根据预测模式refine权重导出模式。当然二者也可以交替实施，如果复杂度允许。模板可以作为实施这些方法的纽带，更进一步地，模板权重可以作为实施这些方法的纽带。即根据预测模式在模板上的预测值，以及权重导出模式对应的子模板或模板权重，来计算模板上的代价。利用模板上的代价来推导或者说确定预测模式或权重导出模式。
也就是说，在本申请实施例中，根据K个初始预测模式，确定当前块的权重导出模式，再使用当前块的权重导出模式对K个初始预测模式进行改善，得到K个调整的预测模式，进而根据当前块的权重导出模式和K个调整的预测模式，确定当前块的预测值。
其中，上述根据K个调整的预测模式和当前块的权重导出模式，确定当前块的预测值的具体过程可以参照下面根据权重导出模式和K个预测模式对当前块进行预测，确定当前块的预测值的具体描述内容，在此不再赘述。
下面通过具体的实施例，对上述S202中根据K个初始预测模式确定当前块的权重导出模式的过程，以及上述S203-A中根据当前块的权重导出模式，确定K个调整的预测模式的过程进行详细介绍。
在一些实施例中，上述S202包括如下S202-A至S202-C的步骤：
S202-A、确定N个候选权重导出模式，N为大于1的正整数；
S202-B、针对N个候选权重导出模式中的第i个候选权重导出模式，根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价，i为从1到N的正整数；
S202-C、根据N个候选权重导出模式分别对应的第一代价，确定当前块的权重导出模式。
在该实施例中，编码端首先确定N个候选权重导出模式，该N个候选权重导出模式构成权重导出模式的列表，编码端通过计算这N个候选权重导出模式中每个候选权重导出模式对应的第一代价，根据第一代价，从这N个候选权重导出模式中，确定出当前块的权重导出模式。
例如，将N个候选权重导出模式中第一代价最小的候选权重导出模式，确定为当前块的权重导出模式。
再例如，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表；根据候选权重导出模式列表，确定当前块的权重导出模式。例如，将候选权重导出模式列表中的任意一个候选权重导出模式确定为当前块的权重导出模式。在该实现方式中，编码端将当前块的权重导出模式对应的权重导出模式索引写入码流，以使解码端根据该权重导出模式索引，从确定的候选权重导出模式列表中，确定出当前块的权重导出模式。
可选的，上述第一代价可以为SAD、SATD或SEE等代价。
编码端确定N个候选权重导出模式中每个候选权重导出模式对应的第一代价的过程相同，为了便于描述，以确定N个候选权重导出模式中的第i个候选权重导出模式对应的第一代价为例。
上述S202-B中根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价的方式包括但不限于如下几种：
方式一，使用第i个候选权重导出模式和K个初始预测模式对当前块的模板进行预测，得到模板的第一预测值；根据模板的第一预测值和模板的重建值，确定第i个权重导出模式对应的第一代价。
具体的，根据第i个候选权重导出模式确定模板权重，使用K个初始预测模式对模板进行预测，得到模板的K个预测值，使用模板权重对模板对应的K个预测值进行加权处理，得到模板的预测值，为了便于描述，将该模板的预测值记为模板的第一预测值。接着，根据模板的第一预测值和模板的重建值，确定该第i个候选权重导出模式关于模板的预测代价，将该预测代价记为第i个权重导出模式对应的第一代价。
需要说明的是，根据根据第i个候选权重导出模式确定模板权重的具体过程可以参照下面根据权重导出模式，确定模板权重的具体内容描述，在此不再赘述。
根据上述方法，可以确定出N个候选权重导出模式中每个候选权重导出模式对应的第一代价，进而将N个候选权重导出模式中第一代价最小的候选权重导出模式确定为当前块的权重导出模式，或者，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表，将候选权重导出模式列表中权重导出模式索引对应候选权重导出模式，确定为当前块的权重导出模式。
在该方式一的基础上，上述S203-A中根据当前块的权重导出模式，确定K个调整的预测模式包括如下步骤：
S203-A1、根据当前块的权重导出模式，对K个初始预测模式进行改善，得到K个调整的预测模式。
其中，S203-A1的实现方式包括但不限于如下几种示例：
示例1，确定与当前块的权重导出模式相关的至少一个预测模式，这至少一个预测模式可以是预测角度与当前块的权重导出模式的分界线平行或近似平行的预测模式或者为预测角度与当前块的权重导出模式的分界面垂值或近似垂直的预测模式。将K个初始预测模式中的至少一个初始预测模式，使用与当前块的权重导出模式相关的至少一个预测模式中的预测模式进行替换，形成多个新的预测模式组合，每个新的预测模式组合中包括K个预测模式，针对每个新的预测模式组合，使用该新的预测模式组合所包括的K个预测模式和当前块的权重导出模式对目标进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该新的预测模式组合对应的代价，进而可以获得上述形成的多个新的预测模式组合中每一个新的预测模式组合对应的代价。同理，使用当前块的权重导出模式和K个初始预测模式对当前块进行预测，确定K个初始预测模式对应的代价。这样，编码端可以根据新的预测模式组合对应的代价，以及K个初始预测模式对应的代价，得到K个调整的预测模式，例如若新的预测模式组合1对应的代价最小，则将该新的预测模式组合1所包括的K个预测模式，确定为K个调整的预测模式，该新的预测模式组合1中可能包括至少一个初始预测模式，也可能不包括初始预测模式。
示例2，通过如下S203-A11至S203-A14的步骤对K个初始预测模式进行改善，得到K个调整的预测模式：
S203-A11、根据当前块的权重导出模式，确定第一模板权重；
其中，根据当前块的权重导出模式，确定第一模板权重的具体过程可以参照下述模板权重的导出实施例的具体描述，在此不再赘述。
S203-A12、针对K个初始预测模式中的第j个初始预测模式，确定第j个初始预测模式对应的M个第一候选预测模式，M为大于1的正整数，j为从1到K的正整数。
需要说明的是，上述S203-A12与上述S203-A11在执行过程中没有先后顺序，即上述S203-A12可以在上述S203-A11之后执行，也可以在S203-A11之前执行，或者与上述S203-A11同步执行，本申请实施例对此不做限制。
在该示例中，编码端使用当前块的权重导出模式对K个初始预测模式中每个初始预测模式进行改善，得到K个初始预测模式中每个初始预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
其中，确定K个初始预测模式中每个初始预测模式对应的调整的预测模式的过程相同，为了便于描述，本申请实施例以确定K个初始预测模式中第j个初始预测模式对应的调整的预测模式为例进行说明。
本申请实施例中，编码端针对第j个初始预测模式，首先确定该第j个初始预测模式对应的M个第一候选预测模式。
编码端确定第j个初始预测模式对应的M个第一候选预测模式的方式包括如下两种情况：
情况1，若第j个初始预测模式为帧内预测模式，将与第j个初始预测模式相似的M个帧内预测模式，确定为M个第一候选预测模式。例如，第j个初始预测模式为角度预测模式，则将与第j个初始预测模式的角度相似(或相近)的M个角度预测模式，确定第j个初始预测模式对应的M个第一候选预测模式。
情况2，若第j个初始预测模式为帧间预测模式，则根据第j个初始预测模式进行搜索，得到M个第一运动信息，根据这M个第一运动信息，确定第j个预测模式对应的M个第一候选预测模式。例如，对于第j个初始预测模式，其运动矢量是(xInit，yInit)，设置一个搜索范围如水平方向从xInit-sR到xInit+sR，竖直方向从yInit-sR到yInit+sR的矩形区域，其中sR可以是2，4，8等。该矩形区域内的每一个运动矢量都可以和第j个初始预测模式的其他信息，如参考帧索引和预测列表标志等组合起来确定一个运动信息，从而确定一个预测模式，这样可以获得第j个预测模式对应的M个第一候选预测模式。
在一些实施例中，上述根据第j个初始预测模式进行搜索，得到M个第一运动信息，包括：根据第i个初始预测模式进行搜索，得到M个偏移量；根据M个偏移量，得到M个第一运动信息。
S203-A13、根据第一模板权重，确定使用M个第一候选预测模式分别对模板进行预测时所对应的第二代价。
根据上述情况1或情况2的方法，确定出第j个初始预测模式对应的M个第一候选预测模式后，确定这M个第一候选预测模式对应的代价，进而根据代价确定第j个初始预测模式对应的调整的预测模式。为了便于描述，将第一候选预测模式对应的代价，记为第二代价。
具体的，针对M个第一候选预测模式中的每个第一候选预测模式，使用该第一候选预测模式对当前块的模板进行预测，得到一个预测值，接着，根据该第一模板权重和该预测值确定该第一候选预测模式关于模板的第二代价。
下面以计算M个第一候选预测模式中一个第一候选预测模式关于模板的第二代价为例进行说明。
为了便于描述，以模板中的第i个像素点为例，该第i个像素点可以理解为模板中的任意一个点，也就是说，确定模板中每个点的第二代价的过程相同，参照第i个像素点即可。具体是，使用该第一候选预测模式对模板进行预测，得到模板的预测值，将第i个像素点在模板的预测值中对应的预测值记为第i个预测值，将该第i个像素点在模板的重建值中对应的重建值记为第i个重建值，进而根据第i个预测值和第i个重建值，以及第i个像素点在第一模板权重中的权重，确定该第一候选预测模式在第i个像素点的第二代价。根据上述方法，确定出该第一候选预测模式在模板中的每个点或多个点处的第二代价，进而根据模板中每个点或多个点的第二代价，确定出第一候选预测模式关于模板的第二代价。例如，将该第一候选预测模式在模板中像素点处的第二代价之和，确定为该第一候选预测模式关于模板的第二代价。
示例性的，以SAD代价为例，可以根据上述公式(1)确定出第一候选预测模式在模板中的第i个像素点(x，y)处的第二代价。
需要说明的是，上述以SAD为例来确定第一候选预测模式关于模板的第二代价，可选的还可以根据SATD、MSE 等代价计算方法，确定第一候选预测模式关于模板的第二代价。
根据上述方法，可以确定出M个第一候选预测模式中每个第一候选预测模式对应的第二代价，接着，执行如下S203-A14。
S203-A14、根据M个第一候选预测模式分别对应的第二代价，确定第j个初始预测模式对应的调整的预测模式。
例如，将M个第一候选预测模式中第二代价最小的第一候选预测模式，确定为第j个初始预测模式对应的调整的预测模式。
上述实施例以确定K个初始预测模式中的第j个初始预测模式对应的调整的预测模式为例进行说明，参照上述描述，可以确定出K个初始预测模式中每个初始预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
在该方式一中，编码端首先根据K个初始预测模式，确定出当前块的权重导出模式，再根据当前块的权重导出模式对K个初始预测模式进行改善，例如通过上述示例1或示例2的方式进行改善，得到K个调整的预测模式，进而使用当前块的权重导出模式和K个调整的预测模式对当前块进行预测，得到当前块的预测值。
本申请实施例中，编码端除了使用上述方式一的方式确定当前块的权重导出模式和K个调整的预测模式外，还可以通过如下方式二，确定当前块的权重导出模式和K个调整的预测模式。
方式二，将预测模式的改善和当前块的权重导出模式的确定融合在一起，同步进行，此时上述S202-B中根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价包括如下S202-B11至S202-B12的步骤：
S202-B11、根据第i个候选权重导出模式，对K个初始预测模式进行改善，得到第i个候选权重导出模式对应的K个第一改善预测模式。
在该方式二中，对于N个候选预测模式中的第i个候选权重导出模式，首先使用该第i个候选权重导出模式对K个初始预测模式进行改善，得到该第i个候选权重导出模式对应的K个改善预测模式，为了便于描述，将此处第i个候选权重导出模式对应的K个改善预测模式记为K个第一改善预测模式。接着该第i个候选权重导出模式，以及该第i个候选权重导出模式对应的K个第一改善预测模式，确定第i个候选权重导出模式对应的第一代价，这样可以得到更精确的当前块的权重导出模式和K个调整的预测模式。
其中，S202-B11的实现方式包括但不限于如下几种示例：
示例1，确定与第i个候选权重导出模式相关的至少一个预测模式，这至少一个预测模式可以是预测角度与第i个候选权重导出模式的分界线平行或近似平行的预测模式或者为预测角度与第i个候选权重导出模式的分界面垂值或近似垂直的预测模式。将K个初始预测模式中的至少一个初始预测模式，使用与第i个候选权重导出模式相关的至少一个预测模式中的预测模式进行替换，形成多个新的预测模式组合，每个新的预测模式组合中包括K个预测模式，针对每个新的预测模式组合，使用该新的预测模式组合所包括的K个预测模式和第i个候选权重导出模式对模板进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该新的预测模式组合对应的代价，进而可以获得上述形成的多个新的预测模式组合中每一个新的预测模式组合对应的代价。同理，使用第i个候选权重导出模式和K个初始预测模式对当前块进行预测，确定K个初始预测模式对应的代价。这样，编码端可以根据新的预测模式组合对应的代价，以及K个初始预测模式对应的代价，得到K个第一改善预测模式，例如若新的预测模式组合1对应的代价最小，则将该新的预测模式组合1所包括的K个预测模式，确定为K个第一改善预测模式，该新的预测模式组合1中可能包括至少一个初始预测模式，也可能不包括初始预测模式。
示例2，通过如下S202-B11-1至S202-B11-4的步骤对对K个初始预测模式进行改善，得到第i个候选权重导出模式对应的K个第一改善预测模式：
S202-B11-1、根据第i个候选权重导出模式，确定第二模板权重。
其中，根据第i个候选权重导出模式，确定第二模板权重的具体过程可以参照下述模板权重的导出实施例的具体描述，在此不再赘述。
S202-B11-2、针对K个初始预测模式中的第j个初始预测模式，确定第j个初始预测模式对应的M个第一候选预测模式，M为正整数，j为从1到K的正整数。
需要说明的是，上述S202-B11-2与上述S202-B11-1在执行过程中没有先后顺序，即上述S202-B11-2可以在上述S202-B11-1之后执行，也可以在S202-B11-1之前执行，或者与上述S202-B11-1同步执行，本申请实施例对此不做限制。
在该示例中，编码端使用第i个候选权重导出模式对K个初始预测模式中每个初始预测模式进行改善，得到第i个候选权重导出模式下，K个初始预测模式中每个初始预测模式对应的第一改善预测模式，进而得到K个第一改善预测模式。
其中，确定K个初始预测模式中每个初始预测模式对应的第一改善预测模式的过程相同，为了便于描述，本申请实施例以确定K个初始预测模式中第j个初始预测模式对应的第一改善预测模式为例进行说明。
本申请实施例中，编码端针对第j个初始预测模式，首先确定该第j个初始预测模式对应的M个第一候选预测模式。
编码端确定第j个初始预测模式对应的M个第一候选预测模式的方式包括如下两种情况：
情况1，若第j个初始预测模式为帧内预测模式，将与第j个初始预测模式相似的M个帧内预测模式，确定为M个第一候选预测模式。
情况2，若第j个初始预测模式为帧间预测模式，则根据第j个初始预测模式进行搜索，得到M个第一运动信息，根据这M个第一运动信息，确定第j个预测模式对应的M个第一候选预测模式。
在一些实施例中，上述根据第j个初始预测模式进行搜索，得到M个第一运动信息，包括：根据第i个初始预测模式进行搜索，得到M个偏移量；根据M个偏移量，得到M个第一运动信息。
S202-B11-3、根据第二模板权重，确定使用M个第一候选预测模式分别对模板进行预测时所对应的第三代价。
根据上述情况1或情况2的方法，确定出第j个初始预测模式对应的M个第一候选预测模式后，确定这M个第一候选预测模式对应的代价，进而根据代价确定第j个初始预测模式对应的第一改善预测模式。为了便于描述，将此时第一候选预测模式对应的代价，记为第三代价。
具体的，针对M个第一候选预测模式中的每个第一候选预测模式，使用该第一候选预测模式对当前块的模板进行预测，得到一个预测值，接着，根据该第二模板权重和该预测值确定该第一候选预测模式关于模板的第三代价。
下面以计算M个第一候选预测模式中一个第一候选预测模式关于模板的第三代价为例进行说明。
为了便于描述，以模板中的第i个像素点为例，该第i个像素点可以理解为模板中的任意一个点，也就是说，确定模板中每个点的第三代价的过程相同，参照第i个像素点即可。具体是，使用该第一候选预测模式对模板进行预测，得到模板的预测值，将第i个像素点在模板的预测值中对应的预测值记为第i个预测值，将该第i个像素点在模板的重建值中对应的重建值记为第i个重建值，进而根据第i个预测值和第i个重建值，以及第i个像素点在第二模板权重中的权重，确定该第一候选预测模式在第i个像素点的第三代价。根据上述方法，确定出该第一候选预测模式在模板中的每个点或多个点处的第三代价，进而根据模板中每个点或多个点的第三代价，确定出第一候选预测模式关于模板的第三代价。例如，将该第一候选预测模式在模板中像素点处的第三代价之和，确定为该第一候选预测模式关于模板的第三代价。
示例性的，可以参照上述公式(1)和(2)确定出第一候选预测模式关于模板的第三代价，具体是，是将上述公式(1)中wTemplateValue[x][y]替换为像素点(x，y)在第二模板权重上对应的权重值。
根据上述方法，可以确定出M个第一候选预测模式中每个第一候选预测模式对应的第三代价，接着，执行如下S202-B11-4。
S202-B11-4、根据M个第一候选预测模式分别对应的第三代价，确定第j个初始预测模式对应的改善预测模式。
例如，将M个第一候选预测模式中所述第三代价最小的第一候选预测模式，确定为第j个初始预测模式对应的第一改善预测模式。
上述实施例以确定K个初始预测模式中的第j个初始预测模式对应的第一改善预测模式为例进行说明，参照上述描述，可以确定出K个初始预测模式中每个初始预测模式对应的第一改善预测模式，进而得到第i个候选权重导出模式下的K个第一改善预测模式。
S202-B12、根据第i个候选权重导出模式，以及K个第一改善预测模式，确定第i个候选权重导出模式对应的第一代价。
示例性的，使用第i个候选权重导出模式和K个第一改善预测模式对当前块的模板进行预测，得到模板的第二预测值；根据模板的第二预测值和模板的重建值，确定第i个权重导出模式对应的第一代价。例如，使用第i个候选权重导出模式导出第二模板权重，使用K个第一改善预测模式对模板进行预测，得到K个预测值，使用第二模板权重对这K个预测值进行加权处理，得到模板的第二预测值，根据该模板的第二预测值与模板的重建值，确定第i个权重导出模式对应的第一代价。
上述以确定第i个候选权重导出模式对应的第一代价为例，针对N个候选权重导出模式中的每个候选权重导出模式，均可以采用上述确定第i个权重导出模式对应的第一代价的方式，确定出N个候选权重导出模式中每个候选权重导出模式对应的第一代价，进而根据N个候选权重导出模式中各候选权重导出模式对应的第一代价，确定当前块的权重导出模式。
例如，将N个候选权重导出模式中第一代价最小的候选权重导出模式，确定为当前块的权重导出模式。
再例如，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表；根据候选权重导出模式列表中权重导出模式索引，确定当前块的权重导出模式。
在该方式二中，确定出了N个候选权重导出模式中，每个候选权重导出模式对应的K个第一改善预测模式，在此基础上，上述S203-A中根据当前块的权重导出模式，确定K个调整的预测模式包括如下步骤：
S203-A2、将当前块的权重导出模式对应的K个第一改善预测模式，确定为K个调整的预测模式。
在该方式二中，由于将预测模式的改善和当前块的权重导出模式的确定融合在一起，即在确定N个候选权重导出模式对应的第一代价时，确定出了每个候选权重导出模式对应的K个第一改善预测模式。这样，根据第一代价确定出当前块的权重导出模式后，可以将该当前块的权重导出模式对应的K个第一改善预测模式，确定为K个调整的预测模式，进而使用当前块的权重导出模式和这K个调整的预测模式对当前块进行预测，得到当前块的预测值。
该方式二中，对于N个候选权重导出模式中的每个候选权重导出模式，均确定出每个候选权重导出模式对应的K个第一改善预测模式，进而根据每个候选权重导出模式及其对应的K个第一改善预测模式，对模板进行预测，确定第一代价，基于该第一代价可以更精确地从N个候选权重导出模式中确定出当前块的权重导出模式。
本申请实施例中，编码端除了使用上述方式一或方式二的方式确定当前块的权重导出模式和K个调整的预测模式外，还可以通过如下方式三，确定当前块的权重导出模式和K个调整的预测模式。
方式三，为了降低计算复杂度，在该方式三中，在对初始预测模式进行改善时，不使用每一个候选权重导出模式对初始预测模式进行改善，而是使用简单的P个子模板，例如当前块的全部模板、左侧模板、上侧模板，对初始预测 模式进行改善，从而减少初始预测模式改善的计算量。此时上述S202-B中根据第i个候选权重导出模式和K个初始预测模式，确定第i个候选权重导出模式对应的第一代价包括如下S202-B21至S202-B23的步骤：
S202-B21、针对K个初始预测模式中的第j个初始预测模式，确定第j个初始预测模式关于P个子模板的第二改善预测模式，P为正整数，j为从1到K的正整数。
在该方式三中，编码端使用P个子模板对K个初始预测模式进行改善，得到K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式。对于N个候选权重导出模式中的每个候选权重导出模式，根据该候选权重导出模式与P个子模板的对应关系，从K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式中，得到该候选权重导出模式对应的K个第二改善预测模式，进而根据该候选权重导出模式及其对应的K个第二预测模式对当前块的模板进行预测，确定该候选权重导出模式对应的第一代价。最后，根据N个候选权重导出模式中每个权重导出模式对应的第一代价，确定当前块的权重导出模式。在方式三中，不再单独使用候选预测模式对K个初始预测模式进行改善，进而降低了初始预测模式的改善复杂度。
示例性的，P个子模板包括当前块的左侧模板TM_L、上侧模板TM_A和全部模板TM_AL中的至少一个，其中全部模板TM_AL包括左侧模板TM_L和上侧模板TM_A。
本申请实施例中，编码端确定K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式的过程相同，为了便于描述，本申请实施例以确定K个初始预测模式中第j个初始预测模式关于P个子模板的第二改善预测模式为例进行说明。
在一种示例中，假设P个子模板包括当前块的左侧模板TM_L、上侧模板TM_A和全部模板TM_AL，则确定第j个初始预测模式关于P个子模板的第二改善预测模式包括：确定第j个初始预测模式关于左侧模板TM_L的第二改善预测模式，确定第j个初始预测模式关于上侧模板TM_A的第二改善预测模式，以及确定第j个初始预测模式关于全部模板TM_AL的第二改善预测模式。也就是说，该步骤中，编码端确定第j个初始预测模式关于P个子模板中每个子模板的第二改善预测模式。
本申请实施例对确定第j个初始预测模式关于P个子模板的第二改善预测模式的具体方式不做限制。
在一些实施例中，S202-B21包括如下S202-B21-1至S202-B21-3的步骤：
S202-B21-1、确定第j个初始预测模式对应的M个第一候选预测模式，M为大于1的正整数。
其中，编码端确定第j个初始预测模式对应的M个第一候选预测模式的方式包括如下两种情况：
情况1，若第j个初始预测模式为帧内预测模式，将与第j个初始预测模式相似的M个帧内预测模式，确定为M个第一候选预测模式。
情况2，若第j个初始预测模式为帧间预测模式，则根据第j个初始预测模式进行搜索，得到M个第一运动信息，根据这M个第一运动信息，确定第j个预测模式对应的M个第一候选预测模式。
在一些实施例中，上述根据第j个初始预测模式进行搜索，得到M个第一运动信息，包括：根据第i个初始预测模式进行搜索，得到M个偏移量；根据M个偏移量，得到M个第一运动信息。
S202-B21-2、针对P个子模板中的第p个子模板，确定使用M个第一候选预测模式分别对第p个子模板进行预测时所对应的第四代价，p为从1到P的正整数。
编码端根据上述情况1或情况2的方法，确定出第j个初始预测模式对应的M个第一候选预测模式后，确定这M个第一候选预测模式分别关于P个子模板中每一个子模板对应的代价，进而根据代价确定第j个初始预测模式关于P个子模板中每一个子模板对应的第二改善预测模式。为了便于描述，将此时第一候选预测模式对应的代价，记为第四代价。
具体的，以P个子模板中的第p个子模板为例，针对M个第一候选预测模式的任一第一候选预测模式，使用该第一候选预测模式对第p个子模板进行预测，得到第p个子模板的预测值；根据第p个子模板的预测值和第p个子模板的重建值，确定第一候选预测模式关于第P个子模板的第四代价。参照该方法，可以确定出M个第一候选预测模式中每一个第一候选预测模式关于P个子模板中每个子模板的第四代价。
举例说明，假设第j个初始预测模式对应3个第一候选预测模式，分别记为第一候选预测模式1、第一候选预测模式2和第一候选预测模式3，假设P个子模板包括左侧模板TM_L、上侧模板TM_A和全部模板TM_AL。首先，对于左侧模板TM_L，使用第一候选预测模式1、第一候选预测模式2和第一候选预测模式3分别对左侧模板TM_L进行预测，得到对应的预测值分别为预测值1、预测值2和预测值3。接着，根据预测值1和左侧模板TM_L的重建值，确定出第一候选预测模式1关于左侧模板TM_L的第四代价，根据预测值2和左侧模板TM_L的重建值，确定出第一候选预测模式2关于左侧模板TM_L的第四代价，根据预测值3和左侧模板TM_L的重建值，确定出第一候选预测模式3关于左侧模板TM_L的第四代价。
同理，对于上侧模板TM_A，使用第一候选预测模式1、第一候选预测模式2和第一候选预测模式3分别对上侧模板TM_A进行预测，得到对应的预测值分别为预测值4、预测值5和预测值6。接着，根据预测值4和左侧模板TM_L的重建值，确定出第一候选预测模式1关于上侧模板TM_A的第四代价，根据预测值5和上侧模板TM_A的重建值，确定出第一候选预测模式2关于上侧模板TM_A的第四代价，根据预测值6和上侧模板TM_A的重建值，确定出第一候选预测模式3关于上侧模板TM_A的第四代价。
同理，对于全部模板TM_AL，使用第一候选预测模式1、第一候选预测模式2和第一候选预测模式3分别对全部 模板TM_AL进行预测，得到对应的预测值分别为预测值7、预测值8和预测值9。接着，根据预测值7和全部模板TM_AL的重建值，确定出第一候选预测模式1关于全部模板TM_AL的第四代价，根据预测值8和全部模板TM_AL的重建值，确定出第一候选预测模式2关于全部模板TM_AL的第四代价，根据预测值9和全部模板TM_AL的重建值，确定出第一候选预测模式3关于全部模板TM_AL的第四代价。
根据上述方法，确定出第j个初始预测模式对应的M个第一候选预测模式中，每一个第一候选预测模式关于P个子模板中每个子模板的第四代价，接着，执行如下S202-B21-3。
S202-B21-3、根据M个第一候选预测模式分别关于第p个子模板的第四代价，确定第j个初始预测模式关于第p个子模板的第二改善预测模式。
例如，将M个第一候选预测模式中关于第p个子模板的最小第四代价对应的第一候选预测模式，确定为第j个初始预测模式关于第p个子模板的第二改善预测模式。
继续参照上述示例，假设第j个初始预测模式对应3个第一候选预测模式，分别记为第一候选预测模式1、第一候选预测模式2和第一候选预测模式3，假设P个子模板包括左侧模板TM_L、上侧模板TM_A和全部模板TM_AL。对于左侧模板TM_L，假设第一候选预测模式1关于上侧模板TM_A的第四代价小于第一候选预测模式2关于上侧模板TM_A的第四代价，第一候选预测模式2关于上侧模板TM_A的第四代价小于第一候选预测模式3关于上侧模板TM_A的第四代价，这样可以将第一候选预测模式1确定为第j个初始预测模式关于左侧模板TM_L的第二改善预测模式。
同理，对于上侧模板TM_A，假设第一候选预测模式2关于上侧模板TM_A的第四代价小于第一候选预测模式1关于上侧模板TM_A的第四代价，第一候选预测模式1关于上侧模板TM_A的第四代价小于第一候选预测模式3关于上侧模板TM_A的第四代价，这样可以将第一候选预测模式2确定为第j个初始预测模式关于上侧模板TM_A的第二改善预测模式。
同理，对于全部模板TM_AL，假设第一候选预测模式2关于全部模板TM_AL的第四代价小于第一候选预测模式1关于全部模板TM_AL的第四代价，第一候选预测模式1关于全部模板TM_AL的第四代价小于第一候选预测模式3关于全部模板TM_AL的第四代价，这样可以将第一候选预测模式2确定为第j个初始预测模式关于全部模板TM_AL的第二改善预测模式。
根据上述方法，可以确定出K个初始预测模式中，每个初始预测模式关于P个子模板中每个子模板的第二改善预测模式。
示例性的，假设K＝2，即2个初始预测模式分别记为第一预测模式和第二预测模式，假设P个子模板为左侧模板TM_L、上侧模板TM_A和全部模板TM_AL，根据上述方法，确定出第一预测模式和第二预测模式中每个初始预测模式关于左侧模板TM_L、上侧模板TM_A和全部模板TM_AL中每个子模板的第二改善预测模式如表6所示。
S202-B22、根据第i个候选权重导出模式，从K个初始预测模式分别关于P个子模板的第二改善预测模式中，确定第i个候选权重导出模式对应的K个第二改善预测模式。
具体的，针对N个候选权重导出模式中的第i个候选权重导出模式，根据该第i个候选权重导出模式所对应的子模板，从K个初始预测模式中每个初始预测模式关于P个子模板的第二改善预测模式中，确定出该第i个候选权重导出模式对应的K个第二改善预测模式。
例如，以K＝2为例，第i个候选权重导出模式对应的子模板为上侧模板TM_A和全部模板TM_AL，参照上述表6，可以确定出第一预测模式关于上侧模板TM_A的第二改善预测模式为第二改善预测模式12，第二预测模式关于全部模板TM_AL的第二改善预测模式为第二改善预测模式23，进而可以确定出第i个候选权重导出模式对应的2个第二改善预测模式分别为第二改善预测模式12和第二改善预测模式23。
在一种可能的实现方式中，编码端确定第i个候选权重导出模式对应的划分角度，根据划分角度和预测模式的模板之间的对应关系(具体参照上述表5所示)，从K个初始预测模式分别关于P个子模板的第二改善预测模式中，确定出第i个候选权重导出模式对应的K个第二改善预测模式。
示例性的，以K＝2为例，根据上述表2，确定出第i个候选权重导出模式对应的角度索引，例如第i个候选权重导出模式为27，则对应的角度索引为12，在上述表5所示的划分角度和预测模式的模板之间的对应关系中，查询该角度索引下，第一预测模式对应的模板和第二预测模式对应的模板，再根据第一预测模式对应的模板和第二预测模式对应的模板在上述表6中，确定出该第i个候选权重导出模式对应的2个第二改善预测模式，例如，从表5中查询出该角度索引下，第一预测模式对应的模板为上侧模板TM_A，第二预测模式对应的模板为全部模板TM_AL，这样从表6中，第一预测模式对应的第二改善预测模式中上侧模板TM_A对应的第二改善预测模式12，以及第二预测模式对应的第二改善预测模式中全部模板TM_AL对应的第二改善预测模式23，确定为第i个候选权重导出模式对应的2个第二改善预测模式。
根据上述方法，确定出N个候选权重导出模式中，第i个候选权重模式对应的K个第二改善预测模式，接着执行如下S202-B23的步骤。
S202-B23、根据第i个候选权重导出模式，以及第i个候选权重导出模式对应的K个第二改善预测模式，确定第i个候选权重导出模式对应的第一代价。
具体的，使用第i个候选权重导出模式和K个第二改善预测模式对当前块的模板进行预测，得到模板的第三预测 值，根据模板的第三预测值和模板的重建值，确定第i个权重导出模式对应的第一代价。例如，根据第i个候选权重导出模式确定模板权重，使用K个第二预测改善模式分别对当前块的模板进行预测，得到K个预测值，使用确定的模板权重对该K个预测值进行加权处理，得到第i个候选权重导出模式对应的模板预测值，进而根据模板预测值和模板的重建值，确定该第i个候选权重导出模式对应的第一代价。
上述以确定第i个候选权重导出模式对应的第一代价为例，针对N个候选权重导出模式中的每个候选权重导出模式，均可以采用上述确定第i个权重导出模式对应的第一代价的方式，确定出N个候选权重导出模式中每个候选权重导出模式对应的第一代价，进而根据N个候选权重导出模式中各候选权重导出模式对应的第一代价，确定当前块的权重导出模式。
例如，将N个候选权重导出模式中第一代价最小的候选权重导出模式，确定为当前块的权重导出模式。
再例如，根据N个候选权重导出模式分别对应的第一代价，对N个候选权重导出模式进行排序，得到候选权重导出模式列表；根据候选权重导出模式列表，确定当前块的权重导出模式。
在该方式三中，使用P个子模板对K个初始预测模式进行改善，得到K个初始预测模式中，每个初始预测模式关于P个子模板中每个子模板的第二改善预测模式。这样，在后期确定N个候选权重导出模式中每个候选权重导出模式对应的第一代价时，不再根据不同的候选权重导出模式对K个初始预测模式分别进行改善，而是统一从上述确定的K个初始预测模式中，每个初始预测模式关于P个子模板中每个子模板的第二改善预测模式中，查询各候选权重导出模式对应的K个第二改善预测模式，进而降低了对初始预测模式进行改善的复杂度，且提高了计算各候选权重导出模式对应的第一代价的速度，从而可以快速从N个候选权重导出模式中确定出当前块的权重导出模式。
在该方式三中，基于P个子模板，确定出了N个候选权重导出模式中，每个候选权重导出模式对应的K个第二改善预测模式，在此基础上，上述S203-A中根据当前块的权重导出模式，确定K个调整的预测模式的方式包括但不限于如下几种：
方式1，将当前块的权重导出模式对应的K个第二改善预测模式，确定为K个调整的预测模式。
上述方式三，在确定N个候选权重导出模式中，每个候选权重导出模式对应的第一代价时，确定出了每个候选权重导出模式对应的K个第二改善预测模式，这样根据第一代价确定出当前块的权重导出模式后，可以直接将该当前块的权重导出模式对应的K个第二改善预测模式，确定为K个调整的预测模式。
方式2，为了进一步提高K个调整的预测模式的精确度，则对当前块的权重导出模式对应的K个第二改善预测模式进行改善，以得到K个调整的预测模式，此时，上述S203-A包括如下步骤：
S203-A3、根据当前块的权重导出模式，对K个第二改善预测模式进行改善，得到K个调整的预测模式。
其中，S203-A3的实现方式包括但不限于如下几种示例：
示例1，确定与当前块的权重导出模式相关的至少一个预测模式，这至少一个预测模式可以是预测角度与当前块的权重导出模式的分界线平行或近似平行的预测模式或者为预测角度与当前块的权重导出模式的分界面垂值或近似垂直的预测模式。将K个第二改善预测模式中的至少一个第二改善预测模式，使用与当前块的权重导出模式相关的至少一个预测模式中的预测模式进行替换，形成多个新的预测模式组合，每个新的预测模式组合中包括K个预测模式，针对每个新的预测模式组合，使用该新的预测模式组合所包括的K个预测模式和当前块的权重导出模式对目标进行预测，得到模板的预测值，根据模板的预测值和模板的重建值，确定该新的预测模式组合对应的代价，进而可以获得上述形成的多个新的预测模式组合中每一个新的预测模式组合对应的代价。同理，使用当前块的权重导出模式和K个第二改善预测模式对当前块进行预测，确定K个第二改善预测模式对应的代价。这样，编码端可以根据新的预测模式组合对应的代价，以及K个第二改善预测模式对应的代价，得到K个调整的预测模式，例如若新的预测模式组合1对应的代价最小，则将该新的预测模式组合1所包括的K个预测模式，确定为K个调整的预测模式，该新的预测模式组合1中可能包括至少一个初始预测模式，也可能不包括初始预测模式。
示例2，通过如下S203-A31至S203-A34的步骤对K个第二预测模式进行改善，得到K个调整的预测模式：
S203-A31、根据当前块的权重导出模式，确定第一模板权重。
其中，根据当前块的权重导出模式，确定第一模板权重的具体过程可以参照下述模板权重的导出实施例的具体描述，在此不再赘述。
S203-A32、针对K个第二改善预测模式中的第k个第二改善预测模式，确定第k个第二改善预测模式对应的P个第二候选预测模式，P为大于1的正整数，k为从1到K的正整数。
需要说明的是，上述S203-A32与上述S203-A31在执行过程中没有先后顺序，即上述S203-A32可以在上述S203-A31之后执行，也可以在S203-A31之前执行，或者与上述S203-A31同步执行，本申请实施例对此不做限制。
在该示例中，编码端使用当前块的权重导出模式对K个第二改善预测模式中每个第二改善预测模式进行改善，得到K个第二改善预测模式中每个第二改善预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
其中，确定K个第二改善预测模式中每个第二改善预测模式对应的调整的预测模式的过程相同，为了便于描述，本申请实施例以确定K个第二改善预测模式中第k个第二改善预测模式对应的调整的预测模式为例进行说明。
本申请实施例中，编码端针对第k个第二改善预测模式，首先确定该第k个第二改善预测模式对应的P个第二候选预测模式。
编码端确定第k个第二改善预测模式对应的P个第二候选预测模式的方式包括如下两种情况：
情况1，若第k个第二改善预测模式为帧内预测模式，将与第k个第二改善预测模式相似的P个帧内预测模式， 确定为P个第二候选预测模式。例如，第k个第二改善预测模式为角度预测模式，则将与第k个第二改善预测模式的角度相似(或相近)的P个角度预测模式，确定第k个第二改善预测模式对应的P个第二候选预测模式。
情况2，若第k个第二改善预测模式为帧间预测模式，则根据第k个第二改善预测模式进行搜索，得到P个第二运动信息，根据这P个第二运动信息，确定第k个第二改善预测模式对应的P个第二候选预测模式。例如，对于第k个第二改善预测模式，其运动矢量是(xInit，yInit)，设置一个搜索范围如水平方向从xInit-sR到xInit+sR，竖直方向从yInit-sR到yInit+sR的矩形区域，其中sR可以是2，4，8等。该矩形区域内的每一个运动矢量都可以和第k个第二改善预测模式的其他信息，如参考帧索引和预测列表标志等组合起来确定一个运动信息，从而确定一个预测模式，这样可以获得第k个第二改善预测模式对应的P个第二候选预测模式。
在一些实施例中，上述根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息，包括：根据第k个第二改善预测模式进行搜索，得到P个偏移量；根据P个偏移量，得到P个第二运动信息。
S203-A33、根据第一模板权重，确定使用P个第二候选预测模式分别对模板进行预测时所对应的第五代价。
根据上述情况1或情况2的方法，确定出第k个第二改善预测模式对应的P个第二候选预测模式后，确定这P个第二候选预测模式对应的代价，进而根据代价确定第k个第二改善预测模式对应的调整的预测模式。为了便于描述，将第二候选预测模式对应的代价，记为第五代价。
具体的，针对P个第二候选预测模式中的每个第二候选预测模式，使用该第二候选预测模式对当前块的模板进行预测，得到一个预测值，接着，根据该第一模板权重和该预测值确定该第一候选预测模式关于模板的第五代价。以模板上的一个像素点为例，使用该第二候选预测模式对该像素点进行预测，得到该像素点的一个预测值，在第一模板权重中确定该像素点对应的模板权重，将该像素点的预测值与模板权重的乘积，确定该第二候选预测模式在该像素点处的预测值。参照该方式，可以得到该第二候选预测模式在模板中每个像素点处的预测值，这些预测值组成该第二候选预测模式对应的模板预测值，根据模板预测值和模板的重建值，确定出该第二候选预测模式对应的第五代价。
根据上述方法，可以确定出P个第二候选预测模式中每个第二候选预测模式对应的第五代价，接着，执行如下S203-A14。
S203-A34、根据P个第二候选预测模式分别对应的第五代价，确定第k个第二改善预测模式对应的调整的预测模式。
例如，将P个第二候选预测模式中第五代价最小的第二候选预测模式，确定为第k个第二改善预测模式对应的调整的预测模式。
上述实施例以确定K个第二改善预测模式中的第k个第二改善预测模式对应的调整的预测模式为例进行说明，参照上述描述，可以确定出K个第二改善预测模式中每个第二改善预测模式对应的调整的预测模式，进而得到K个调整的预测模式。
在该方式2中，编码端首先使用P个子模板对K个初始预测模式进行改善，得到当前块的权重导出模式对应的K个第二改善预测模式，再使用当前块的权重导出模式对当前块的权重导出模式对应的K个第二改善预测模式进行改善，得到精准的K个调整的预测模式，这样使用这K个调整的预测模式和当前块的权重导出模式对当前块进行预测时，可以进一步提高预测的准确性。
本申请实施例，通过上述方法，可以确定出当前块的权重导出模式和K个调整的预测模式，进而使用当前块的权重导出模式和K个调整的预测模式，对当前块进行预测，得到当前块的预测值。
本申请中根据当前块的权重导出模式和K个调整的预测模式，确定当前块的预测值的方式包括但不限于如下几种：
方式一，若在上述根据当前块的权重导出模式确定第一模板权重时，通过当前块的权重导出模式确定出第一模板权重，但未确定出当前块中像素点对应的预测值的权重，在该情况下，则上述S203-B包括如下步骤：
S203-B11、根据K个调整的预测模式，确定K个预测值；
S203-B12、根据当前块的权重导出模式，确定预测值的权重；
S203-B13、根据K个预测值和该预测值的权重，确定当前块的预测值。
需要说明的是，在本申请中，当前块的权重导出模式用于对当前块使用的预测值的权重进行确定。具体地，当前块的权重导出模式可以是导出预测值的权重的模式。对于一个给定长度和宽度的块，每一种权重导出模式可以导出一个预测值的权重矩阵；对于同样大小的块，不同权重导出模式导出的预测值的权重矩阵可以不同。
可以理解的是，在本申请的实施例中，编码端在基于K个调整的预测模式以及预测值的权重，确定当前块的预测值时，可以先根据K个调整的预测模式确定K个预测值；然后利用预测值的权重对K个预测值进行加权平均计算，最终便可以获得当前块的预测值。
在该方式一中，可以理解根据当前块的权重导出模式确定第一模板权重和预测值的权重是两个相互独立的过程，互不干涉。
在一些实施例中，本申请还可以采用下面方式二，确定出当前块的预测值。
方式二，若在上述确定第一模板权重时，通过当前块的权重导出模式确定出模板区域和当前块构成的合并区域中像素点的权重，此时，S203-B包括如下步骤：
S204-B21、根据K个调整的预测模式，确定K个预测值；
S204-B22、根据合并区域中像素点的权重，确定K个预测值的权重；
S204-B23、根据K个预测值和K个预测值的权重，确定当前块的预测值。
在该方式二中，在权重导出时，根据当前块的权重导出模式导出合并区域中像素点的权重，而该合并区域包括当前块和当前块的模板区域，进而将合并区域中当前块对应的权重确定为预测值的权重，将合并区域中模板区域对应的权重确定为第一模板权重。也就是说，在该方式二中，将模板区域和当前块作为一个整体区域，可以一步导出第一模板权重和预测值的权重，减少了权重导出的步骤，提高预测效果。
在一些实施例中，上述预测过程是以像素点为单位进行的，对应的上述权重也为像素点对应的权重。此时，对当前块进行预测时，使用K个调整的预测模式中的每个调整的预测模式对当前块中的某一个像素点A进行预测，得到K个调整的预测模式关于像素点A的K个预测值，根据当前块的权重导出模式确定像素点A的K个预测值的权重，对这K个预测值进行加权，得到像素点A的预测值。对当前块中的每一个像素点执行上述步骤，可以得到当前块中每个像素点的预测值，当前块中每个像素点的预测值构成当前块的预测值。以K＝2为例，使用第一预测模式对当前块中的某一个像素点A进行预测，得到该像素点A的第一预测值，使用第二预测模式对该像素点A进行预测，得到该像素点A的第二预测值，根据像素点A对应的预测值权重，对第一预测值和第二预测值进行加权，得到像素点A的预测值。
在一种示例中，以K＝2为例，若第一预测模式和第二预测模式均为帧内预测模式时，采用第一帧内预测模式进行预测，得到第一预测值，采用第二帧内预测模式进行预测，得到第二预测值，根据预测值的权重对第一预测值和第二预测值进行加权，得到当前块的预测值。例如，采用第一帧内预测模式对像素点A进行预测，得到像素点A的第一预测值，采用第二帧内预测模式对像素点A进行预测，得到像素点A的第二预测值，根据像素点A对应的预测值的权重，对第一预测值和第二预测值进行加权，得到像素点A的预测值。
在一些实施例中，若K个调整的预测模式中的第j个调整的预测模式为帧间预测模式，则上述S203-B根据K个调整的预测模式以及当前块的权重导出模式，确定当前块的预测值包括如下步骤：
S203-B31、根据第j个调整的预测模式，确定运动信息；
S203-B32、根据运动信息，确定第j个预测值；
S203-B33、根据K个调整的预测模式中除第j个调整的预测模式外的其他预测模式，确定K-1个预测值；
S203-B34、根据当前块的权重导出模式，确定K个预测值的权重；
S203-B35、根据第j个预测值、K-1个预测值和预测值的权重，确定当前块的预测值。
以K＝2为例，若第一预测模式为帧内预测模式，第二预测模式为帧间预测模式时，采用帧内预测模式进行预测，得到第一预测值，采用帧间预测模式进行预测，得到第二预测值，根据预测值的权重对第一预测值和第二预测值进行加权，得到当前块的预测值。在该示例中，采用帧内预测模式对当前块中每一个点进行预测，得到当前块中每一个点的预测值，当前块中每一个点的预测值，构成当前块的第一预测值。采用帧间预测模式，确定一运动信息，根据该运动信息确定当前块的最佳匹配块，将该最佳匹配块确定为当前块的第二预测值。针对当前块中每个像素点的预测值的权重，对当前块的第一预测值和第二预测值进行逐点加权运算，得到当前块的预测值。例如，对于当前块中的像素点A，根据像素点A的预测值的权重，将当前块的第一预测值中像素点A对应的第一预测值，与当前块的第二预测值中像素点A对应的第二预测值进行加权，得到像素点A的预测值。
在一些实施例中，若K大于2时，则可以根据当前块的权重导出模式确定K个调整的预测模式中任意两个调整的预测模式对应的预测值的权重，K个调整的预测模式中的其他调整的预测模式对应的预测值的权重可以为预设值。例如，K＝3，第一预测模式和第二预测模式对应的预测值的权重根据当前块的权重导出模式导出，第三预测模式对应的预测值的权重为预设值。在一些实施例中，若K个调整的预测模式对应的预测值的总权重一定，例如为8，则可以根据预设权重比例，来确定K个调整的预测模式各自对应的预测值的权重，假设第三预测模式对应的预测值的权重占整个中预测值的总权重的1/4，则可以确定第三预测模式的预测值的总权重为2，预测值的总权重中的剩下3/4分配给第一预测模式和第二预测模式。示例性的，如果根据当前块的权重导出模式导出第一预测模式对应的预测值的权重3，则确定第一预测模式对应的预测值的权重为(3/4)*3，第二预测模式对应的预测值的权重为第一预测模式的预测值的权重为(3/4)*5。
编码端根据上述方法，确定出当前块的预测值，进而将当前块与预测值的差值，确定为当前块的残差值，对当前块的残差值进行变换，得到变换系数，对变换系数进行量化，得到量化系数，并对量化系数进行编码，得到码流。
本申请实施例提供的预测方法，编码端确定K个初始预测模式；根据所述K个初始预测模式，确定当前块的权重导出模式；根据当前块的权重导出模式，确定当前块的预测值。即本申请通过K个初始预测模式，确定当前块的权重导出模式，进而丰富了权重导出模式的确定方式。另外，本申请中编码端在码流中可以不传输权重导出模式的索引，进而节约了码字，降低了编码代价。
应理解，图14至图19仅为本申请的示例，不应理解为对本申请的限制。
以上结合附图详细描述了本申请的优选实施方式，但是，本申请并不限于上述实施方式中的具体细节，在本申请的技术构思范围内，可以对本申请的技术方案进行多种简单变型，这些简单变型均属于本申请的保护范围。例如，在上述具体实施方式中所描述的各个具体技术特征，在不矛盾的情况下，可以通过任何合适的方式进行组合，为了避免不必要的重复，本申请对各种可能的组合方式不再另行说明。又例如，本申请的各种不同的实施方式之间也可以进行任意组合，只要其不违背本申请的思想，其同样应当视为本申请所公开的内容。
还应理解，在本申请的各种方法实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。另外，本申请实施例中，术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系。具体地，A和/或B可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本申请中字符“/”，一般表示前后关联对象是一种“或”的关系。
上文结合图14至图19，详细描述了本申请的方法实施例，下文结合图20至图22，详细描述本申请的装置实施例。
图20是本申请一实施例提供的预测装置的示意性框图，该预测装置10应用于上述视频解码器。
如图20所示，预测装置10包括：
解码单元11，用于解码码流，确定K个初始预测模式，所述K为大于1的正整数；
确定单元12，用于根据所述K个初始预测模式，确定所述当前块的权重导出模式；
预测单元13，用于根据所述当前块的权重导出模式，确定所述当前块的预测值。
在一些实施例中，预测单元13，具体用于根据所述当前块的权重导出模式，确定K个调整的预测模式；根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值。
在一些实施例中，确定单元12，具体用于确定N个候选权重导出模式，所述N为大于1的正整数；针对所述N个候选权重导出模式中的第i个候选权重导出模式，根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，所述i为从1到N的正整数；根据所述N个候选权重导出模式分别对应的第一代价，确定所述当前块的权重导出模式。
在一些实施例中，确定单元12，具体用于使用所述第i个候选权重导出模式和所述K个初始预测模式对所述当前块的模板进行预测，得到所述模板的第一预测值；根据所述模板的第一预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
在一些实施例中，确定单元12，具体用于根据所述当前块的权重导出模式，对所述K个初始预测模式进行改善，得到所述K个调整的预测模式。
在一些实施例中，确定单元12，具体用于根据所述当前块的权重导出模式，确定第一模板权重；针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数，所述j为从1到K的正整数；根据所述第一模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第二代价；根据所述M个第一候选预测模式分别对应的第二代价，确定所述第j个初始预测模式对应的调整的预测模式。
在一些实施例中，确定单元12，具体用于将所述M个第一候选预测模式中所述第二代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的调整的预测模式。
在一些实施例中，确定单元12，具体用于根据所述第i个候选权重导出模式，对所述K个初始预测模式进行改善，得到所述第i个候选权重导出模式对应的K个第一改善预测模式；根据所述第i个候选权重导出模式，以及所述K个第一改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
在一些实施例中，确定单元12，具体用于根据所述第i个候选权重导出模式，确定第二模板权重；针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为正整数，所述j为从1到K的正整数；根据所述第二模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第三代价；根据所述M个第一候选预测模式分别对应的第三代价，确定所述第j个初始预测模式对应的第一改善预测模式。
在一些实施例中，确定单元12，具体用于将所述M个第一候选预测模式中所述第三代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的第一改善预测模式。
在一些实施例中，确定单元12，具体用于使用所述第i个候选权重导出模式和所述K个第一改善预测模式对所述当前块的模板进行预测，得到所述模板的第二预测值；根据所述模板的第二预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
在一些实施例中，确定单元12，具体用于将所述当前块的权重导出模式对应的K个第一改善预测模式，确定为所述K个调整的预测模式。
在一些实施例中，确定单元12，具体用于针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式关于P个子模板的第二改善预测模式，所述P为正整数，所述j为从1到K的正整数；根据所述第i个候选权重导出模式，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定所述第i个候选权重导出模式对应的K个第二改善预测模式；根据所述第i个候选权重导出模式，以及所述K个第二改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
在一些实施例中，确定单元12，具体用于确定所述第i个候选权重导出模式对应的划分角度；根据所述划分角度和预测模式的模板之间的对应关系，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定出所述第i个候选权重导出模式对应的K个第二改善预测模式。
可选的，所述P个子模板包括所述当前块的左侧模板、上侧模板和全部模板中的至少一个。
在一些实施例中，确定单元12，具体用于确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数；针对P个子模板中的第p个子模板，确定使用所述M个第一候选预测模式分别对所述第p个子模板进行预测时所对应的第四代价，所述p为从1到P的正整数；根据所述M个第一候选预测模式分别关于所述第p个子模板的第四代价，确定所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
在一些实施例中，确定单元12，具体用于将所述M个第一候选预测模式中关于所述第p个子模板的最小第四代价对应的第一候选预测模式，确定为所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
在一些实施例中，确定单元12，具体用于针对所述M个第一候选预测模式的任一第一候选预测模式，使用所述第一候选预测模式对所述第p个子模板进行预测，得到所述第p个子模板的预测值；根据所述第p个子模板的预测值和所述第p个子模板的重建值，确定所述第一候选预测模式关于所述第P个子模板的第四代价。
在一些实施例中，确定单元12，具体用于使用所述第i个候选权重导出模式和所述K个第二改善预测模式对所述当前块的模板进行预测，得到所述模板的第三预测值；根据所述模板的第三预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
在一些实施例中，确定单元12，具体用于根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式。
在一些实施例中，确定单元12，具体用于将所述当前块的权重导出模式对应的K个第二改善预测模式，确定为所述K个调整的预测模式。
在一些实施例中，确定单元12，具体用于根据所述当前块的权重导出模式，对所述K个第二改善预测模式进行改善，得到所述K个调整的预测模式。
在一些实施例中，确定单元12，具体用于根据所述当前块的权重导出模式，确定第一模板权重；针对所述K个第二改善预测模式中的第k个第二改善预测模式，确定所述第k个第二改善预测模式对应的P个第二候选预测模式，所述P为大于1的正整数，所述k为从1到K的正整数；根据所述第一模板权重，确定使用所述P个第二候选预测模式分别对所述模板进行预测时所对应的第五代价；根据所述P个第二候选预测模式分别对应的第五代价，确定所述第k个第二改善预测模式对应的调整的预测模式。
在一些实施例中，确定单元12，具体用于将所述P个第二候选预测模式中所述第五代价最小的第二候选预测模式，确定为所述第k个第二改善预测模式对应的调整的预测模式。
在一些实施例中，若所述第j个初始预测模式为帧内预测模式，则确定单元12，具体用于将与所述第j个初始预测模式相似的M个帧内预测模式，确定为所述M个第一候选预测模式。
在一些实施例中，若所述第j个初始预测模式为帧间预测模式，则确定单元12，具体用于根据所述第j个初始预测模式进行搜索，得到M个第一运动信息；根据所述M个第一运动信息，得到所述M个第一候选预测模式。
在一些实施例中，确定单元12，具体用于根据所述第i个初始预测模式进行搜索，得到M个偏移量；根据所述M个偏移量，得到所述M个第一运动信息。
在一些实施例中，确定单元12，具体用于将所述第k个第二改善预测模式相似的P个帧内预测模式，确定为所述P个第二候选预测模式。
在一些实施例中，确定单元12，具体用于根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息；根据所述P个第二运动信息，得到所述P个第二候选预测模式。
在一些实施例中，确定单元12，具体用于根据所述第k个第二改善预测模式进行搜索，得到P个偏移量；根据所述P个偏移量，得到P个第二运动信息。
在一些实施例中，确定单元12，具体用于将所述N个候选权重导出模式中所述第一代价最小的候选权重导出模式，确定为所述当前块的权重导出模式。
在一些实施例中，确定单元12，具体用于根据所述N个候选权重导出模式分别对应的第一代价，对所述N个候选权重导出模式进行排序，得到候选权重导出模式列表；解码所述码流，得到权重导出模式索引；将所述候选权重导出模式列表中所述权重导出模式索引对应候选权重导出模式，确定为所述当前块的权重导出模式。
在一些实施例中，解码单元11，具体用于确定备选预测模式列表，所述备选预测模式列表中包括至少两个备选预测模式；解码所述码流，得到预测模式索引；根据所述预测模式索引，从所述备选预测模式列表中，确定出所述K个初始预测模式。
在一些实施例中，确定单元12，具体用于根据权重导出模式，确定角度索引和距离索引；根据所述角度索引、所述距离索引和模板的大小，确定模板权重；若所述权重导出模式为所述当前块的权重导出模式时，则所述模板权重为所述第一模板权重，若所述权重导出模式为所述第i个候选权重导出模式，则所述模板权重为所述第二模板权重。
在一些实施例中，确定单元12，具体用于根据所述角度索引、所述距离索引和所述模板的大小，确定所述模板中像素点的第一参数，所述第一参数用于确定权重；根据所述模板中像素点的第一参数，确定所述模板中像素点的权重；根据所述模板中像素点的权重，确定所述模板权重。
在一些实施例中，预测单元13，具体用于根据所述K个调整的预测模式，确定K个预测值；根据所述当前块的权重导出模式，确定所述K个预测值的权重；根据所述K个预测模式和所述K个预测值的权重，确定所述当前块的预测值。
在一些实施例中，确定单元12，具体用于根据所述角度索引、所述距离索引、所述模板的大小和所述当前块的大小，确定所述当前块和所述模板组成的合并区域中像素点的权重；根据所述模板的大小和所述合并区域中像素点的权重，确定所述模板权重。
在一些实施例中，预测单元12，具体用于根据所述K个调整的预测模式，确定K个预测值；根据所述合并区域 中像素点的权重，确定所述K个预测值的权重；根据所述K个预测值和所述K个预测值的权重，确定所述当前块的预测值。
在一些实施例中，预测单元12，具体用于根据所述合并区域中所述当前块的像素点对应的权重，确定所述K个预测值的权重。
可选的，所述模板包括所述当前块的上方已解码区域、左侧已解码区域和左上方已解码区域中的至少一个。
应理解，装置实施例与方法实施例可以相互对应，类似的描述可以参照方法实施例。为避免重复，此处不再赘述。具体地，图20所示的装置10可以执行本申请实施例的解码端的预测方法，并且装置10中的各个单元的前述和其它操作和/或功能分别为了实现上述解码端的预测方法等各个方法中的相应流程，为了简洁，在此不再赘述。
图21是本申请一实施例提供的预测装置的示意性框图，该预测装置应用于上述编码器。
如图21所示，该预测装置20可以包括：
第一确定单元21，用于确定K个初始预测模式，所述K为大于1的正整数；
第二确定单元22，用于根据所述K个初始预测模式，确定当前块的权重导出模式；
预测单元23，用于根据所述当前块的权重导出模式，确定当前块的预测值。
在一些实施例中，预测单元23，具体用于根据所述当前块的权重导出模式，确定K个调整的预测模式；根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值。
在一些实施例中，第二确定单元22，具体用于确定N个候选权重导出模式，所述N为大于1的正整数；针对所述N个候选权重导出模式中的第i个候选权重导出模式，根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，所述i为从1到N的正整数；根据所述N个候选权重导出模式分别对应的第一代价，确定所述当前块的权重导出模式。
在一些实施例中，第二确定单元22，具体用于使用所述第i个候选权重导出模式和所述K个初始预测模式对所述当前块的模板进行预测，得到所述模板的第一预测值；根据所述模板的第一预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
在一些实施例中，第二确定单元22，具体用于根据所述当前块的权重导出模式，对所述K个初始预测模式进行改善，得到所述K个调整的预测模式。
在一些实施例中，第二确定单元22，具体用于根据所述当前块的权重导出模式，确定第一模板权重；针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数，所述j为从1到K的正整数；根据所述第一模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第二代价；根据所述M个第一候选预测模式分别对应的第二代价，确定所述第j个初始预测模式对应的调整的预测模式。
在一些实施例中，第二确定单元22，具体用于将所述M个第一候选预测模式中所述第二代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的调整的预测模式。
在一些实施例中，第二确定单元22，具体用于根据所述第i个候选权重导出模式，对所述K个初始预测模式进行改善，得到所述第i个候选权重导出模式对应的K个第一改善预测模式；根据所述第i个候选权重导出模式，以及所述K个第一改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
在一些实施例中，第二确定单元22，具体用于根据所述第i个候选权重导出模式，确定第二模板权重；针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为正整数，所述j为从1到K的正整数；根据所述第二模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第三代价；根据所述M个第一候选预测模式分别对应的第三代价，确定所述第j个初始预测模式对应的第一改善预测模式。
在一些实施例中，第二确定单元22，具体用于将所述M个第一候选预测模式中所述第三代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的第一改善预测模式。
在一些实施例中，第二确定单元22，具体用于使用所述第i个候选权重导出模式和所述K个第一改善预测模式对所述当前块的模板进行预测，得到所述模板的第二预测值；根据所述模板的第二预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
在一些实施例中，第二确定单元22，具体用于将所述当前块的权重导出模式对应的K个第一改善预测模式，确定为所述K个调整的预测模式。
在一些实施例中，第二确定单元22，具体用于针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式关于P个子模板的第二改善预测模式，所述P为正整数，所述j为从1到K的正整数；根据所述第i个候选权重导出模式，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定所述第i个候选权重导出模式对应的K个第二改善预测模式；根据所述第i个候选权重导出模式，以及所述K个第二改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
在一些实施例中，第二确定单元22，具体用于确定所述第i个候选权重导出模式对应的划分角度；根据所述划分角度和预测模式的模板之间的对应关系，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定出所述第i个候选权重导出模式对应的K个第二改善预测模式。
可选的，所述P个子模板包括所述当前块的左侧模板、上侧模板和全部模板中的至少一个。
在一些实施例中，第二确定单元22，具体用于确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数；针对P个子模板中的第p个子模板，确定使用所述M个第一候选预测模式分别对所述第p个子模板进行预测时所对应的第四代价，所述p为从1到P的正整数；根据所述M个第一候选预测模式分别关于所述第p个子模板的第四代价，确定所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
在一些实施例中，第二确定单元22，具体用于将所述M个第一候选预测模式中关于所述第p个子模板的最小第四代价对应的第一候选预测模式，确定为所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
在一些实施例中，第二确定单元22，具体用于针对所述M个第一候选预测模式的任一第一候选预测模式，使用所述第一候选预测模式对所述第p个子模板进行预测，得到所述第p个子模板的预测值；根据所述第p个子模板的预测值和所述第p个子模板的重建值，确定所述第一候选预测模式关于所述第P个子模板的第四代价。
在一些实施例中，第二确定单元22，具体用于使用所述第i个候选权重导出模式和所述K个第二改善预测模式对所述当前块的模板进行预测，得到所述模板的第三预测值；根据所述模板的第三预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
在一些实施例中，第二确定单元22，具体用于根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式。
在一些实施例中，第二确定单元22，具体用于将所述当前块的权重导出模式对应的K个第二改善预测模式，确定为所述K个调整的预测模式。
在一些实施例中，第二确定单元22，具体用于根据所述当前块的权重导出模式，对所述K个第二改善预测模式进行改善，得到所述K个调整的预测模式。
在一些实施例中，第二确定单元22，具体用于根据所述当前块的权重导出模式，确定第一模板权重；针对所述K个第二改善预测模式中的第k个第二改善预测模式，确定所述第k个第二改善预测模式对应的P个第二候选预测模式，所述P为大于1的正整数，所述k为从1到K的正整数；根据所述第一模板权重，确定使用所述P个第二候选预测模式分别对所述模板进行预测时所对应的第五代价；根据所述P个第二候选预测模式分别对应的第五代价，确定所述第k个第二改善预测模式对应的调整的预测模式。
在一些实施例中，第二确定单元22，具体用于将所述P个第二候选预测模式中所述第五代价最小的第二候选预测模式，确定为所述第k个第二改善预测模式对应的调整的预测模式。
在一些实施例中，若所述第j个初始预测模式为帧内预测模式，则第二确定单元22，具体用于将与所述第j个初始预测模式相似的M个帧内预测模式，确定为所述M个第一候选预测模式。
在一些实施例中，若所述第j个初始预测模式为帧间预测模式，则第二确定单元22，具体用于根据所述第j个初始预测模式进行搜索，得到M个第一运动信息；根据所述M个第一运动信息，得到所述M个第一候选预测模式。
在一些实施例中，第二确定单元22，具体用于根据所述第i个初始预测模式进行搜索，得到M个偏移量；根据所述M个偏移量，得到所述M个第一运动信息。
在一些实施例中，若所述第k个第二改善预测模式为帧内预测模式，则第二确定单元22，具体用于将所述第k个第二改善预测模式相似的P个帧内预测模式，确定为所述P个第二候选预测模式。
在一些实施例中，若所述第k个第二改善预测模式为帧间预测模式，则第二确定单元22，具体用于根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息；根据所述P个第二运动信息，得到所述P个第二候选预测模式。
在一些实施例中，第二确定单元22，具体用于根据所述第k个第二改善预测模式进行搜索，得到P个偏移量；根据所述P个偏移量，得到P个第二运动信息。
在一些实施例中，第二确定单元22，具体用于将所述N个候选权重导出模式中所述第一代价最小的候选权重导出模式，确定为所述当前块的权重导出模式。
在一些实施例中，第二确定单元22，具体用于根据所述N个候选权重导出模式分别对应的第一代价，对所述N个候选权重导出模式进行排序，得到候选权重导出模式列表；根据所述候选权重导出模式列表，确定所述当前块的权重导出模式。
在一些实施例中，第一确定单元22，还用于将所述候选权重导出模式列表中所述当前块的权重导出模式对应的权重导出模式索引，写入所述码流。
在一些实施例中，第一确定单元22，还用于在码流中写入预测模式索引，所述预测模式索引用于确定所述K个初始预测模式。
在一些实施例中，第二确定单元22，具体用于根据权重导出模式，确定角度索引和距离索引；根据所述角度索引、 所述距离索引和模板的大小，确定模板权重；若所述权重导出模式为所述当前块的权重导出模式时，则所述模板权重为所述第一模板权重，若所述权重导出模式为所述第i个候选权重导出模式，则所述模板权重为所述第二模板权重。
在一些实施例中，第二确定单元22，具体用于根据所述角度索引、所述距离索引和所述模板的大小，确定所述模板中像素点的第一参数，所述第一参数用于确定权重；根据所述模板中像素点的第一参数，确定所述模板中像素点的权重；根据所述模板中像素点的权重，确定所述模板权重。
在一些实施例中，预测单元23，具体用于根据所述K个调整的预测模式，确定K个预测值；根据所述当前块的权重导出模式，确定所述K个预测值的权重；根据所述K个预测模式和所述K个预测值的权重，确定所述当前块的预测值。
在一些实施例中，第二确定单元22，具体用于根据所述角度索引、所述距离索引、所述模板的大小和所述当前块的大小，确定所述当前块和所述模板组成的合并区域中像素点的权重；根据所述模板的大小和所述合并区域中像素点的权重，确定所述模板权重。
在一些实施例中，预测单元23，具体用于根据所述K个调整的预测模式，确定K个预测值；根据所述合并区域中像素点的权重，确定所述K个预测值的权重；根据所述K个预测值和所述K个预测值的权重，确定所述当前块的预测值。
在一些实施例中，预测单元23，具体用于根据所述合并区域中所述当前块的像素点对应的权重，确定所述K个预测值的权重。
可选的，所述模板包括所述当前块的上方已解码区域、左侧已解码区域和左上方已解码区域中的至少一个。
应理解，装置实施例与方法实施例可以相互对应，类似的描述可以参照方法实施例。为避免重复，此处不再赘述。具体地，图21所示的装置20可以对应于执行本申请实施例的编码端的预测方法中的相应主体，并且装置20中的各个单元的前述和其它操作和/或功能分别为了实现编码端的预测方法等各个方法中的相应流程，为了简洁，在此不再赘述。
上文中结合附图从功能单元的角度描述了本申请实施例的装置和系统。应理解，该功能单元可以通过硬件形式实现，也可以通过软件形式的指令实现，还可以通过硬件和软件单元组合实现。具体地，本申请实施例中的方法实施例的各步骤可以通过处理器中的硬件的集成逻辑电路和/或软件形式的指令完成，结合本申请实施例公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件单元组合执行完成。可选地，软件单元可以位于随机存储器，闪存、只读存储器、可编程只读存储器、电可擦写可编程存储器、寄存器等本领域的成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法实施例中的步骤。
图22是本申请实施例提供的电子设备的示意性框图。
如图22所示，该电子设备30可以为本申请实施例所述的视频编码器，或者视频解码器，该电子设备30可包括：
存储器33和处理器32，该存储器33用于存储计算机程序34，并将该程序代码34传输给该处理器32。换言之，该处理器32可以从存储器33中调用并运行计算机程序34，以实现本申请实施例中的方法。
例如，该处理器32可用于根据该计算机程序34中的指令执行上述方法200中的步骤。
在本申请的一些实施例中，该处理器32可以包括但不限于：
通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等等。
在本申请的一些实施例中，该存储器33包括但不限于：
易失性存储器和/或非易失性存储器。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDR SDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DR RAM)。
在本申请的一些实施例中，该计算机程序34可以被分割成一个或多个单元，该一个或者多个单元被存储在该存储器33中，并由该处理器32执行，以完成本申请提供的方法。该一个或多个单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述该计算机程序34在该电子设备30中的执行过程。
如图22所示，该电子设备30还可包括：
收发器33，该收发器33可连接至该处理器32或存储器33。
其中，处理器32可以控制该收发器33与其他设备进行通信，具体地，可以向其他设备发送信息或数据，或接收其他设备发送的信息或数据。收发器33可以包括发射机和接收机。收发器33还可以进一步包括天线，天线的数量可以为一个或多个。
应当理解，该电子设备30中的各个组件通过总线系统相连，其中，总线系统除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。
图23是本申请实施例提供的视频编解码系统的示意性框图。
如图23所示，该视频编解码系统40可包括：视频编码器41和视频解码器42，其中视频编码器41用于执行本申请实施例涉及的视频编码方法，视频解码器42用于执行本申请实施例涉及的视频解码方法。
本申请还提供了一种计算机存储介质，其上存储有计算机程序，该计算机程序被计算机执行时使得该计算机能够执行上述方法实施例的方法。或者说，本申请实施例还提供一种包含指令的计算机程序产品，该指令被计算机执行时使得计算机执行上述方法实施例的方法。
本申请还提供了一种码流，该码流是根据上述编码方法生成的，可选的，该码流中包括上述第一标志，或者包括第一标志和第二标志。
当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行该计算机程序指令时，全部或部分地产生按照本申请实施例该的流程或功能。该计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。该计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，该计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。该计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。该可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如数字视频光盘(digital video disc，DVD))、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以意识到，结合本申请中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，该单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。例如，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
以上内容，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以该权利要求的保护范围为准。

Claims (87)

  1. 一种预测方法，应用于解码器，其特征在于，包括：
    解码码流，确定K个初始预测模式，所述K为大于1的正整数；
    根据所述K个初始预测模式，确定当前块的权重导出模式；
    根据所述当前块的权重导出模式，确定所述当前块的预测值。
  2. 根据权利要求1所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定所述当前块的预测值，包括：
    根据所述当前块的权重导出模式，确定K个调整的预测模式；
    根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值。
  3. 根据权利要求2所述的方法，其特征在于，所述根据所述K个初始预测模式，确定当前块的权重导出模式，包括：
    确定N个候选权重导出模式，所述N为大于1的正整数；
    针对所述N个候选权重导出模式中的第i个候选权重导出模式，根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，所述i为从1到N的正整数；
    根据所述N个候选权重导出模式分别对应的第一代价，确定所述当前块的权重导出模式。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    使用所述第i个候选权重导出模式和所述K个初始预测模式对所述当前块的模板进行预测，得到所述模板的第一预测值；
    根据所述模板的第一预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，对所述K个初始预测模式进行改善，得到所述K个调整的预测模式。
  6. 根据权利要求5所述的方法，其特征在于，所述根据所述当前块的权重导出模式，对所述K个初始预测模式进行改善，得到所述K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，确定第一模板权重；
    针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数，所述j为从1到K的正整数；
    根据所述第一模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第二代价；
    根据所述M个第一候选预测模式分别对应的第二代价，确定所述第j个初始预测模式对应的调整的预测模式。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述M个第一候选预测模式中各第一候选预测模式对应的第二代价，确定所述第j个初始预测模式对应的调整的预测模式，包括：
    将所述M个第一候选预测模式中所述第二代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的调整的预测模式。
  8. 根据权利要求3所述的方法，其特征在于，所述根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    根据所述第i个候选权重导出模式，对所述K个初始预测模式进行改善，得到所述第i个候选权重导出模式对应的K个第一改善预测模式；
    根据所述第i个候选权重导出模式，以及所述K个第一改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
  9. 根据权利要求8所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，对所述K个初始预测模式进行改善，得到所述第i个候选权重导出模式对应的K个第一改善预测模式，包括：
    根据所述第i个候选权重导出模式，确定第二模板权重；
    针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为正整数，所述j为从1到K的正整数；
    根据所述第二模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第三代价；
    根据所述M个第一候选预测模式分别对应的第三代价，确定所述第j个初始预测模式对应的第一改善预测模式。
  10. 根据权利要求9所述的方法，其特征在于，所述根据所述M个第一候选预测模式分别对应的第三代价，确定所述第j个初始预测模式对应的第一改善预测模式，包括：
    将所述M个第一候选预测模式中所述第三代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的第一改善预测模式。
  11. 根据权利要求8所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，以及所述K个第一改善预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    使用所述第i个候选权重导出模式和所述K个第一改善预测模式对所述当前块的模板进行预测，得到所述模板的第二预测值；
    根据所述模板的第二预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
  12. 根据权利要求8所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定K个调整的预测模式，包括：
    将所述当前块的权重导出模式对应的K个第一改善预测模式，确定为所述K个调整的预测模式。
  13. 根据权利要求3所述的方法，其特征在于，所述根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式关于P个子模板的第二改善预测模式，所述P为正整数，所述j为从1到K的正整数；
    根据所述第i个候选权重导出模式，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定所述第i个候选权重导出模式对应的K个第二改善预测模式；
    根据所述第i个候选权重导出模式，以及所述K个第二改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
  14. 根据权利要求13所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定所述第i个候选权重导出模式对应的K个第二改善预测模式，包括：
    确定所述第i个候选权重导出模式对应的划分角度；
    根据所述划分角度和预测模式的模板之间的对应关系，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定出所述第i个候选权重导出模式对应的K个第二改善预测模式。
  15. 根据权利要求13所述的方法，其特征在于，所述P个子模板包括所述当前块的左侧模板、上侧模板和全部模板中的至少一个。
  16. 根据权利要求15所述的方法，其特征在于，所述确定所述第j个初始预测模式关于P个子模板的第二改善预测模式，包括：
    确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数；
    针对P个子模板中的第p个子模板，确定使用所述M个第一候选预测模式分别对所述第p个子模板进行预测时所对应的第四代价，所述p为从1到P的正整数；
    根据所述M个第一候选预测模式分别关于所述第p个子模板的第四代价，确定所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
  17. 根据权利要求16所述的方法，其特征在于，所述根据所述M个第一候选预测模式分别关于所述第p个子模板的第四代价，确定所述第j个初始预测模式关于第p个子模板的第二改善预测模式，包括：
    将所述M个第一候选预测模式中关于所述第p个子模板的最小第四代价对应的第一候选预测模式，确定为所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
  18. 根据权利要求16所述的方法，其特征在于，所述确定使用所述M个第一候选预测模式分别对所述第p个子模板进行预测时所对应的第四代价，包括：
    针对所述M个第一候选预测模式的任一第一候选预测模式，使用所述第一候选预测模式对所述第p个子模板进行预测，得到所述第p个子模板的预测值；
    根据所述第p个子模板的预测值和所述第p个子模板的重建值，确定所述第一候选预测模式关于所述第P个子模板的第四代价。
  19. 根据权利要求13所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，以及所述K个第二改善预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    使用所述第i个候选权重导出模式和所述K个第二改善预测模式对所述当前块的模板进行预测，得到所述模板的第三预测值；
    根据所述模板的第三预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
  20. 根据权利要求13所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定K个调整的预测模式，包括：
    根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式。
  21. 根据权利要求20所述的方法，其特征在于，所述根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式，包括：
    将所述当前块的权重导出模式对应的K个第二改善预测模式，确定为所述K个调整的预测模式。
  22. 根据权利要求20所述的方法，其特征在于，所述根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，对所述K个第二改善预测模式进行改善，得到所述K个调整的预测模式。
  23. 根据权利要求22所述的方法，其特征在于，所述根据所述当前块的权重导出模式，对所述K个第二改善预测模式进行改善，得到所述K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，确定第一模板权重；
    针对所述K个第二改善预测模式中的第k个第二改善预测模式，确定所述第k个第二改善预测模式对应的P个第二候选预测模式，所述P为大于1的正整数，所述k为从1到K的正整数；
    根据所述第一模板权重，确定使用所述P个第二候选预测模式分别对所述模板进行预测时所对应的第五代价；
    根据所述P个第二候选预测模式分别对应的第五代价，确定所述第k个第二改善预测模式对应的调整的预测模式。
  24. 根据权利要求23所述的方法，其特征在于，所述根据所述P个第二候选预测模式分别对应的第五代价，确定所述第k个第二改善预测模式对应的调整的预测模式，包括：
    将所述P个第二候选预测模式中所述第五代价最小的第二候选预测模式，确定为所述第k个第二改善预测模式对应的调整的预测模式。
  25. 根据权利要求6、10或16所述的方法，其特征在于，若所述第j个初始预测模式为帧内预测模式，则所述确定所述第j个初始预测模式对应的M个第一候选预测模式，包括：
    将与所述第j个初始预测模式相似的M个帧内预测模式，确定为所述M个第一候选预测模式。
  26. 根据权利要求6、10或16所述的方法，其特征在于，若所述第j个初始预测模式为帧间预测模式，则所述确定所述第j个初始预测模式对应的M个第一候选预测模式，包括：
    根据所述第j个初始预测模式进行搜索，得到M个第一运动信息；
    根据所述M个第一运动信息，得到所述M个第一候选预测模式。
  27. 根据权利要求26所述的方法，其特征在于，所述根据所述第j个初始预测模式进行搜索，得到M个第一运动信息，包括：
    根据所述第i个初始预测模式进行搜索，得到M个偏移量；
    根据所述M个偏移量，得到所述M个第一运动信息。
  28. 根据权利要求23所述的方法，其特征在于，若所述第k个第二改善预测模式为帧内预测模式，则所述确定所述第k个第二改善预测模式对应的P个第二候选预测模式，包括：
    将所述第k个第二改善预测模式相似的P个帧内预测模式，确定为所述P个第二候选预测模式。
  29. 根据权利要求23所述的方法，其特征在于，若所述第k个第二改善预测模式为帧间预测模式，则所述确定所述第k个第二改善预测模式对应的P个第二候选预测模式，包括：
    根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息；
    根据所述P个第二运动信息，得到所述P个第二候选预测模式。
  30. 根据权利要求29所述的方法，其特征在于，所述根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息，包括：
    根据所述第k个第二改善预测模式进行搜索，得到P个偏移量；
    根据所述P个偏移量，得到P个第二运动信息。
  31. 根据权利要求3-24任一项所述的方法，其特征在于，所述根据所述N个候选权重导出模式分别对应的代价，确定所述当前块的权重导出模式，包括：
    将所述N个候选权重导出模式中所述第一代价最小的候选权重导出模式，确定为所述当前块的权重导出模式。
  32. 根据权利要求3-24任一项所述的方法，其特征在于，所述根据所述N个候选权重导出模式分别对应的第一代价，确定所述当前块的权重导出模式，包括：
    根据所述N个候选权重导出模式分别对应的第一代价，对所述N个候选权重导出模式进行排序，得到候选权重导出模式列表；
    解码所述码流，得到权重导出模式索引；
    将所述候选权重导出模式列表中所述权重导出模式索引对应候选权重导出模式，确定为所述当前块的权重导出模式。
  33. 根据权利要求1-24任一项所述的方法，其特征在于，所述解码码流，确定K个初始预测模式，包括：
    确定备选预测模式列表，所述备选预测模式列表中包括至少两个备选预测模式；
    解码所述码流，得到预测模式索引；
    根据所述预测模式索引，从所述备选预测模式列表中，确定出所述K个初始预测模式。
  34. 根据权利要求6、9或23所述的方法，其特征在于，所述方法还包括：
    根据权重导出模式，确定角度索引和距离索引；
    根据所述角度索引、所述距离索引和模板的大小，确定模板权重；
    若所述权重导出模式为所述当前块的权重导出模式时，则所述模板权重为第一模板权重，若所述权重导出模式为 第i个候选权重导出模式，则所述模板权重为第二模板权重。
  35. 根据权利要求34所述的方法，其特征在于，所述根据所述角度索引、所述距离索引和模板的大小，确定所述模板权重，包括：
    根据所述角度索引、所述距离索引和所述模板的大小，确定所述模板中像素点的第一参数，所述第一参数用于确定权重；
    根据所述模板中像素点的第一参数，确定所述模板中像素点的权重；
    根据所述模板中像素点的权重，确定所述模板权重。
  36. 根据权利要求35所述的方法，其特征在于，所述根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值，包括：
    根据所述K个调整的预测模式，确定K个预测值；
    根据所述当前块的权重导出模式，确定预测值的权重；
    根据所述K个预测模式和所述预测值的权重，确定所述当前块的预测值。
  37. 根据权利要求34所述的方法，其特征在于，所述根据所述角度索引、所述距离索引和模板的大小，确定所述模板权重，包括：
    根据所述角度索引、所述距离索引、所述模板的大小和所述当前块的大小，确定所述当前块和所述模板组成的合并区域中像素点的权重；
    根据所述模板的大小和所述合并区域中像素点的权重，确定所述模板权重。
  38. 根据权利要求37所述的方法，其特征在于，所述根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值，包括：
    根据所述K个调整的预测模式，确定K个预测值；
    根据所述合并区域中像素点的权重，确定预测值的权重；
    根据所述K个预测值和所述预测值的权重，确定所述当前块的预测值。
  39. 根据权利要求38所述的方法，其特征在于，所述根据所述合并区域中像素点的权重，确定所述K个预测值的权重，包括：
    根据所述合并区域中所述当前块的像素点对应的权重，确定所述K个预测值的权重。
  40. 根据权利要求4-24任一项所述的方法，其特征在于，所述当前块的模板包括所述当前块的上方已解码区域、左侧已解码区域和左上方已解码区域中的至少一个。
  41. 一种预测方法，其特征在于，包括：
    确定K个初始预测模式，所述K为大于1的正整数；
    根据所述K个初始预测模式，确定当前块的权重导出模式；
    根据所述当前块的权重导出模式，确定当前块的预测值。
  42. 根据权利要求41所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定所述当前块的预测值，包括：
    根据所述当前块的权重导出模式，确定K个调整的预测模式；
    根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值。
  43. 根据权利要求42所述的方法，其特征在于，所述根据所述K个初始预测模式，确定所述当前块的权重导出模式，包括：
    确定N个候选权重导出模式，所述N为大于1的正整数；
    针对所述N个候选权重导出模式中的第i个候选权重导出模式，根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，所述i为从1到N的正整数；
    根据所述N个候选权重导出模式分别对应的第一代价，确定所述当前块的权重导出模式。
  44. 根据权利要求43所述的方法，其特征在于，所述根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    使用所述第i个候选权重导出模式和所述K个初始预测模式对所述当前块的模板进行预测，得到所述模板的第一预测值；
    根据所述模板的第一预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
  45. 根据权利要求44所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，对所述K个初始预测模式进行改善，得到所述K个调整的预测模式。
  46. 根据权利要求45所述的方法，其特征在于，所述根据所述当前块的权重导出模式，对所述K个初始预测模式进行改善，得到所述K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，确定第一模板权重；
    针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数，所述j为从1到K的正整数；
    根据所述第一模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第二代价；
    根据所述M个第一候选预测模式分别对应的第二代价，确定所述第j个初始预测模式对应的调整的预测模式。
  47. 根据权利要求46所述的方法，其特征在于，所述根据所述M个第一候选预测模式中各第一候选预测模式对应的第二代价，确定所述第j个初始预测模式对应的调整的预测模式，包括：
    将所述M个第一候选预测模式中所述第二代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的调整的预测模式。
  48. 根据权利要求43所述的方法，其特征在于，所述根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    根据所述第i个候选权重导出模式，对所述K个初始预测模式进行改善，得到所述第i个候选权重导出模式对应的K个第一改善预测模式；
    根据所述第i个候选权重导出模式，以及所述K个第一改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
  49. 根据权利要求48所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，对所述K个初始预测模式进行改善，得到所述第i个候选权重导出模式对应的K个第一改善预测模式，包括：
    根据所述第i个候选权重导出模式，确定第二模板权重；
    针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为正整数，所述j为从1到K的正整数；
    根据所述第二模板权重，确定使用所述M个第一候选预测模式分别对所述模板进行预测时所对应的第三代价；
    根据所述M个第一候选预测模式分别对应的第三代价，确定所述第j个初始预测模式对应的第一改善预测模式。
  50. 根据权利要求49所述的方法，其特征在于，所述根据所述M个第一候选预测模式分别对应的第三代价，确定所述第j个初始预测模式对应的第一改善预测模式，包括：
    将所述M个第一候选预测模式中所述第三代价最小的第一候选预测模式，确定所述第j个初始预测模式对应的第一改善预测模式。
  51. 根据权利要求47所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，以及所述K个第一改善预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    使用所述第i个候选权重导出模式和所述K个第一改善预测模式对所述当前块的模板进行预测，得到所述模板的第二预测值；
    根据所述模板的第二预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
  52. 根据权利要求48所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定K个调整的预测模式，包括：
    将所述当前块的权重导出模式对应的K个第一改善预测模式，确定为所述K个调整的预测模式。
  53. 根据权利要求43所述的方法，其特征在于，所述根据所述第i个候选权重导出模式和所述K个初始预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    针对所述K个初始预测模式中的第j个初始预测模式，确定所述第j个初始预测模式关于P个子模板的第二改善预测模式，所述P为正整数，所述j为从1到K的正整数；
    根据所述第i个候选权重导出模式，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定所述第i个候选权重导出模式对应的K个第二改善预测模式；
    根据所述第i个候选权重导出模式，以及所述K个第二改善预测模式，确定所述第i个候选权重导出模式对应的第一代价。
  54. 根据权利要求53所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定所述第i个候选权重导出模式对应的K个第二改善预测模式，包括：
    确定所述第i个候选权重导出模式对应的划分角度；
    根据所述划分角度和预测模式的模板之间的对应关系，从所述K个初始预测模式分别关于所述P个子模板的第二改善预测模式中，确定出所述第i个候选权重导出模式对应的K个第二改善预测模式。
  55. 根据权利要求53所述的方法，其特征在于，所述P个子模板包括所述当前块的左侧模板、上侧模板和全部模板中的至少一个。
  56. 根据权利要求55所述的方法，其特征在于，所述确定所述第j个初始预测模式关于P个子模板的第二改善预测模式，包括：
    确定所述第j个初始预测模式对应的M个第一候选预测模式，所述M为大于1的正整数；
    针对P个子模板中的第p个子模板，确定使用所述M个第一候选预测模式分别对所述第p个子模板进行预测时所对应的第四代价，所述p为从1到P的正整数；
    根据所述M个第一候选预测模式分别关于所述第p个子模板的第四代价，确定所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
  57. 根据权利要求56所述的方法，其特征在于，所述根据所述M个第一候选预测模式分别关于所述第p个子模板的第四代价，确定所述第j个初始预测模式关于第p个子模板的第二改善预测模式，包括：
    将所述M个第一候选预测模式中关于所述第p个子模板的最小第四代价对应的第一候选预测模式，确定为所述第j个初始预测模式关于第p个子模板的第二改善预测模式。
  58. 根据权利要求56所述的方法，其特征在于，所述确定使用所述M个第一候选预测模式分别对所述第p个子模板进行预测时所对应的第四代价，包括：
    针对所述M个第一候选预测模式的任一第一候选预测模式，使用所述第一候选预测模式对所述第p个子模板进行预测，得到所述第p个子模板的预测值；
    根据所述第p个子模板的预测值和所述第p个子模板的重建值，确定所述第一候选预测模式关于所述第P个子模板的第四代价。
  59. 根据权利要求53所述的方法，其特征在于，所述根据所述第i个候选权重导出模式，以及所述K个第二改善预测模式，确定所述第i个候选权重导出模式对应的第一代价，包括：
    使用所述第i个候选权重导出模式和所述K个第二改善预测模式对所述当前块的模板进行预测，得到所述模板的第三预测值；
    根据所述模板的第三预测值和所述模板的重建值，确定所述第i个权重导出模式对应的第一代价。
  60. 根据权利要求53所述的方法，其特征在于，所述根据所述当前块的权重导出模式，确定K个调整的预测模式，包括：
    根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式。
  61. 根据权利要求60所述的方法，其特征在于，所述根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式，包括：
    将所述当前块的权重导出模式对应的K个第二改善预测模式，确定为所述K个调整的预测模式。
  62. 根据权利要求60所述的方法，其特征在于，所述根据所述当前块的权重导出模式对应的K个第二改善预测模式，确定所述K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，对所述K个第二改善预测模式进行改善，得到所述K个调整的预测模式。
  63. 根据权利要求62所述的方法，其特征在于，所述根据所述当前块的权重导出模式，对所述K个第二改善预测模式进行改善，得到所述K个调整的预测模式，包括：
    根据所述当前块的权重导出模式，确定第一模板权重；
    针对所述K个第二改善预测模式中的第k个第二改善预测模式，确定所述第k个第二改善预测模式对应的P个第二候选预测模式，所述P为大于1的正整数，所述k为从1到K的正整数；
    根据所述第一模板权重，确定使用所述P个第二候选预测模式分别对所述模板进行预测时所对应的第五代价；
    根据所述P个第二候选预测模式分别对应的第五代价，确定所述第k个第二改善预测模式对应的调整的预测模式。
  64. 根据权利要求63所述的方法，其特征在于，所述根据所述P个第二候选预测模式分别对应的第五代价，确定所述第k个第二改善预测模式对应的调整的预测模式，包括：
    将所述P个第二候选预测模式中所述第五代价最小的第二候选预测模式，确定为所述第k个第二改善预测模式对应的调整的预测模式。
  65. 根据权利要求46、50或56所述的方法，其特征在于，若所述第j个初始预测模式为帧内预测模式，则所述确定所述第j个初始预测模式对应的M个第一候选预测模式，包括：
    将与所述第j个初始预测模式相似的M个帧内预测模式，确定为所述M个第一候选预测模式。
  66. 根据权利要求46、50或56所述的方法，其特征在于，若所述第j个初始预测模式为帧间预测模式，则所述确定所述第j个初始预测模式对应的M个第一候选预测模式，包括：
    根据所述第j个初始预测模式进行搜索，得到M个第一运动信息；
    根据所述M个第一运动信息，得到所述M个第一候选预测模式。
  67. 根据权利要求66所述的方法，其特征在于，所述根据所述第j个初始预测模式进行搜索，得到M个第一运动信息，包括：
    根据所述第i个初始预测模式进行搜索，得到M个偏移量；
    根据所述M个偏移量，得到所述M个第一运动信息。
  68. 根据权利要求63所述的方法，其特征在于，若所述第k个第二改善预测模式为帧内预测模式，则所述确定所述第k个第二改善预测模式对应的P个第二候选预测模式，包括：
    将所述第k个第二改善预测模式相似的P个帧内预测模式，确定为所述P个第二候选预测模式。
  69. 根据权利要求63所述的方法，其特征在于，若所述第k个第二改善预测模式为帧间预测模式，则所述确定所述第k个第二改善预测模式对应的P个第二候选预测模式，包括：
    根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息；
    根据所述P个第二运动信息，得到所述P个第二候选预测模式。
  70. 根据权利要求69所述的方法，其特征在于，所述根据所述第k个第二改善预测模式进行搜索，得到P个第二运动信息，包括：
    根据所述第k个第二改善预测模式进行搜索，得到P个偏移量；
    根据所述P个偏移量，得到P个第二运动信息。
  71. 根据权利要求43-64任一项所述的方法，其特征在于，所述根据所述N个候选权重导出模式分别对应的代价，确定所述当前块的权重导出模式，包括：
    将所述N个候选权重导出模式中所述第一代价最小的候选权重导出模式，确定为所述当前块的权重导出模式。
  72. 根据权利要求43-64任一项所述的方法，其特征在于，所述根据所述N个候选权重导出模式分别对应的第一代价，确定所述当前块的权重导出模式，包括：
    根据所述N个候选权重导出模式分别对应的第一代价，对所述N个候选权重导出模式进行排序，得到候选权重导出模式列表；
    根据所述候选权重导出模式列表，确定所述当前块的权重导出模式。
  73. 根据权利要求71所述的方法，其特征在于，所述方法还包括：
    将所述候选权重导出模式列表中所述当前块的权重导出模式对应的权重导出模式索引，写入码流。
  74. 根据权利要求41所述的方法，其特征在于，所述方法还包括：
    在码流中写入预测模式索引，所述预测模式索引用于确定所述K个初始预测模式。
  75. 根据权利要求46、49或63所述的方法，其特征在于，所述方法还包括：
    根据权重导出模式，确定角度索引和距离索引；
    根据所述角度索引、所述距离索引和模板的大小，确定模板权重；
    若所述权重导出模式为所述当前块的权重导出模式时，则所述模板权重为第一模板权重，若所述权重导出模式为第i个候选权重导出模式，则所述模板权重为第二模板权重。
  76. 根据权利要求75所述的方法，其特征在于，所述根据所述角度索引、所述距离索引和模板的大小，确定所述模板权重，包括：
    根据所述角度索引、所述距离索引和所述模板的大小，确定所述模板中像素点的第一参数，所述第一参数用于确定权重；
    根据所述模板中像素点的第一参数，确定所述模板中像素点的权重；
    根据所述模板中像素点的权重，确定所述模板权重。
  77. 根据权利要求76所述的方法，其特征在于，所述根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值，包括：
    根据所述K个调整的预测模式，确定K个预测值；
    根据所述当前块的权重导出模式，确定个预测值的权重；
    根据所述K个预测模式和预测值的权重，确定所述当前块的预测值。
  78. 根据权利要求75所述的方法，其特征在于，所述根据所述角度索引、所述距离索引和模板的大小，确定所述模板权重，包括：
    根据所述角度索引、所述距离索引、所述模板的大小和所述当前块的大小，确定所述当前块和所述模板组成的合并区域中像素点的权重；
    根据所述模板的大小和所述合并区域中像素点的权重，确定所述模板权重。
  79. 根据权利要求78所述的方法，其特征在于，所述根据所述K个调整的预测模式和所述当前块的权重导出模式，确定所述当前块的预测值，包括：
    根据所述K个调整的预测模式，确定K个预测值；
    根据所述合并区域中像素点的权重，确定预测值的权重；
    根据所述K个预测值和预测值的权重，确定所述当前块的预测值。
  80. 根据权利要求79所述的方法，其特征在于，所述根据所述合并区域中像素点的权重，确定所述K个预测值的权重，包括：
    根据所述合并区域中所述当前块的像素点对应的权重，确定所述K个预测值的权重。
  81. 根据权利要求41-64任一项所述的方法，其特征在于，所述模板包括所述当前块的上方已解码区域、左侧已解码区域和左上方已解码区域中的至少一个。
  82. 一种预测装置，其特征在于，包括：
    解码单元，用于解码码流，确定K个初始预测模式，所述K为大于1的正整数；
    确定单元，用于根据所述K个初始预测模式，确定所述当前块的权重导出模式；
    预测单元，用于根据所述当前块的权重导出模式，确定所述当前块的预测值。
  83. 一种预测装置，其特征在于，包括：
    第一确定单元，用于确定K个初始预测模式，所述K为大于1的正整数；
    第二确定单元，用于根据所述K个初始预测模式，确定当前块的权重导出模式；
    预测单元，用于根据所述当前块的权重导出模式，确定当前块的预测值。
  84. 一种电子设备，其特征在于，包括处理器和存储器；
    所示存储器用于存储计算机程序；
    所述处理器用于调用并运行所述存储器中存储的计算机程序，以实现上述权利要求1至40或41至81任一项所述的方法。
  85. 一种视频编解码系统，其特征在于，包括：视频编码器和视频解码器；
    所述的视频解码器用于实现上述权利要求1至40任一项所述的方法；
    所述的视频编码器用于实现上述权利要求41至81任一项所述的方法。
  86. 一种计算机可读存储介质，其特征在于，用于存储计算机程序；
    所述计算机程序使得计算机执行如上述权利要求1至40或41至81任一项所述的方法。
  87. 一种码流，其特征在于，所述码流是基于如上述权利要求41至81任一项所述的方法生成的。
PCT/CN2022/086453 2021-12-31 2022-04-12 预测方法、装置、设备、系统、及存储介质 WO2023123736A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202280084379.1A CN118402239A (zh) 2021-12-31 2022-04-12 预测方法、装置、设备、系统、及存储介质

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
PCT/CN2021/144008 WO2023123495A1 (zh) 2021-12-31 2021-12-31 预测方法、装置、设备、系统、及存储介质
CNPCT/CN2021/144008 2021-12-31

Publications (1)

Publication Number Publication Date
WO2023123736A1 true WO2023123736A1 (zh) 2023-07-06

Family

ID=86997160

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/CN2021/144008 WO2023123495A1 (zh) 2021-12-31 2021-12-31 预测方法、装置、设备、系统、及存储介质
PCT/CN2022/086453 WO2023123736A1 (zh) 2021-12-31 2022-04-12 预测方法、装置、设备、系统、及存储介质

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/144008 WO2023123495A1 (zh) 2021-12-31 2021-12-31 预测方法、装置、设备、系统、及存储介质

Country Status (2)

Country Link
CN (2) CN118414825A (zh)
WO (2) WO2023123495A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111294590A (zh) * 2018-12-06 2020-06-16 华为技术有限公司 用于多假设编码的加权预测方法及装置
CN111385569A (zh) * 2018-12-28 2020-07-07 杭州海康威视数字技术股份有限公司 一种编解码方法及其设备
CN111741297A (zh) * 2020-06-12 2020-10-02 浙江大华技术股份有限公司 帧间预测方法、视频编码方法及其相关装置
CN112055203A (zh) * 2020-08-22 2020-12-08 浙江大华技术股份有限公司 帧间预测方法、视频编码方法及其相关装置
CN113545074A (zh) * 2019-03-21 2021-10-22 北京字节跳动网络技术有限公司 组合帧间帧内预测的改进的加权处理
CN113766245A (zh) * 2020-06-05 2021-12-07 Oppo广东移动通信有限公司 帧间预测方法、解码器、编码器及计算机存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018054269A1 (en) * 2016-09-22 2018-03-29 Mediatek Inc. Method and apparatus for video coding using decoder side intra prediction derivation
WO2019198997A1 (ko) * 2018-04-11 2019-10-17 엘지전자 주식회사 인트라 예측에 기반한 영상 코딩 방법 및 그 장치
WO2020141816A1 (ko) * 2018-12-31 2020-07-09 한국전자통신연구원 영상 부호화/복호화 방법, 장치 및 비트스트림을 저장한 기록 매체
CN113497942A (zh) * 2020-04-08 2021-10-12 Oppo广东移动通信有限公司 一种帧间预测方法、编码器、解码器及存储介质
EP4162688A1 (en) * 2020-06-03 2023-04-12 Nokia Technologies Oy A method, an apparatus and a computer program product for video encoding and video decoding

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111294590A (zh) * 2018-12-06 2020-06-16 华为技术有限公司 用于多假设编码的加权预测方法及装置
CN111385569A (zh) * 2018-12-28 2020-07-07 杭州海康威视数字技术股份有限公司 一种编解码方法及其设备
CN113545074A (zh) * 2019-03-21 2021-10-22 北京字节跳动网络技术有限公司 组合帧间帧内预测的改进的加权处理
CN113766245A (zh) * 2020-06-05 2021-12-07 Oppo广东移动通信有限公司 帧间预测方法、解码器、编码器及计算机存储介质
CN111741297A (zh) * 2020-06-12 2020-10-02 浙江大华技术股份有限公司 帧间预测方法、视频编码方法及其相关装置
CN112055203A (zh) * 2020-08-22 2020-12-08 浙江大华技术股份有限公司 帧间预测方法、视频编码方法及其相关装置

Also Published As

Publication number Publication date
CN118414825A (zh) 2024-07-30
CN118402239A (zh) 2024-07-26
WO2023123495A1 (zh) 2023-07-06

Similar Documents

Publication Publication Date Title
CN110933424A (zh) 一个帧内编码块的多个预测块
JP7540051B2 (ja) 画像予測方法、機器、及びシステム、装置、及び記憶媒体
CN110944198A (zh) 色度模式帧内编码
WO2023044868A1 (zh) 视频编解码方法、设备、系统、及存储介质
WO2023123736A1 (zh) 预测方法、装置、设备、系统、及存储介质
WO2022116105A1 (zh) 视频编解码方法与系统、及视频编码器与视频解码器
WO2023197183A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2023123478A1 (zh) 预测方法、装置、设备、系统、及存储介质
WO2024007128A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2024077553A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2024108391A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2024152254A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
CN111669583A (zh) 图像预测方法、装置、设备、系统及存储介质
WO2023122968A1 (zh) 帧内预测方法、设备、系统、及存储介质
WO2023220970A1 (zh) 视频编码方法、装置、设备、系统、及存储介质
US20240364860A1 (en) Method for video encoding/decoding and bitstream transmission
WO2024216632A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2023197229A1 (zh) 视频编解码方法、装置、设备、系统及存储介质
WO2024183007A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2023122969A1 (zh) 帧内预测方法、设备、系统、及存储介质
WO2024192733A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
TW202433936A (zh) 視訊編解碼方法、裝置、設備、系統、及儲存媒介
WO2024207434A1 (zh) 视频编解码方法、装置、设备、系统、及存储介质
WO2023044919A1 (zh) 视频编解码方法、设备、系统、及存储介质
WO2022155922A1 (zh) 视频编解码方法与系统、及视频编码器与视频解码器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22913038

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE