WO2021035717A1

WO2021035717A1 - Intra-frame chroma prediction method and apparatus, device, and video coding and decoding system

Info

Publication number: WO2021035717A1
Application number: PCT/CN2019/103800
Authority: WO
Inventors: 朱林卫; 张云; 李娜; 张欢; 乔宇
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2019-08-30
Filing date: 2019-08-30
Publication date: 2021-03-04

Abstract

Disclosed in the embodiments of the present application are an intra-frame chroma prediction method and apparatus, a system, a terminal device, a video encoder, a video decoder and a computer-readable storage medium. Said method comprises: acquiring encoded or decoded reconstructed luminance components; down-sampling the encoded or decoded reconstructed luminance components; inputting preset parameters into an image coloring sub-network in a pre-trained chroma prediction convolutional neural network model, so as to obtain chroma components output by the image coloring sub-network, the preset parameters comprising down-sampled encoded or decoded reconstructed luminance components, or comprising down-sampled encoded or decoded reconstructed luminance components and target parameters, and the target parameters comprising at least one of encoding distortions and encoded or decoded reconstructed adjacent chroma blocks; and obtaining a chroma prediction result according to the chroma components. The intra-frame chroma prediction solution based on a convolutional neural network provided by the embodiments of the present application has high universality, and can save code rate.

Description

Intra-frame chrominance prediction method, device, equipment and video coding and decoding system

Technical field

This application belongs to the field of video coding technology, and in particular relates to an intra-frame chrominance prediction method, device, video coding and decoding system, terminal equipment, video encoder, video decoder, and computer-readable storage medium.

Background technique

The video coding process mainly includes modules such as prediction, transform quantization, and entropy coding. Prediction can be divided into intra-frame prediction and inter-frame prediction, and intra-frame prediction can include intra-frame chroma prediction and intra-frame luminance prediction.

At present, in the new generation of video coding standard Versatile Video Coding (VVC), in order to eliminate redundant information in the YCbCr color space, the linear correlation between the luminance component and the chrominance component in the coding block is generally used , Adopt the corresponding linear prediction model CCLM (Based Cross-component Linear Model Chroma Intra-prediction for Video Coding) or multi-model linear prediction model MMLM (Multi-model Based Cross-component Linear Model Chroma Intra-prediction for Video Coding, MM- CCLM) performs intra-frame chroma prediction. However, the existing intra-frame chrominance prediction method cannot be applied to all situations and requires a lot of bit rate.

technical problem

The embodiment of the present application is to provide an intra-frame chrominance prediction method, system, terminal equipment, and device, aiming to solve the problem that the existing intra-frame chrominance prediction method has low universality and requires more code rate.

Technical solutions

In the first aspect, an embodiment of the present application provides an intra-frame chroma prediction method, including:

Obtain the encoded or decoded and reconstructed luminance component; down-sampling the encoded or decoded and reconstructed luminance component; input the preset parameters into the pre-trained chrominance prediction convolutional neural network model of the image on the color Network to obtain the chrominance components output by the color sub-network on the image; wherein, the preset parameters include down-sampled encoded or decoded reconstructed luminance components, or down-sampled encoded or decoded reconstructions The target parameter includes at least one of the encoding distortion and the adjacent chrominance block that has been encoded or decoded; the target chrominance component block is cut out from the chrominance component, and the target chrominance component block is cut out from the chrominance component. The target chrominance component block is the final chrominance prediction result.

In a second aspect, an embodiment of the present application provides an intra-frame chrominance prediction method, which is applied to a video encoder, and the method includes:

Encode the luminance component to obtain the luminance code stream; obtain the encoded and reconstructed luminance component, the adjacent chrominance information that has been encoded and reconstructed, and the original chrominance information corresponding to the chrominance block to be encoded; The target chrominance prediction method with the smallest rate-distortion cost is determined in the degree prediction method; wherein the at least two chrominance prediction methods include a first type of chrominance prediction method and a second type of chrominance prediction method, and the first type The chrominance prediction method is the intra-frame chrominance prediction method according to any one of claims 1 to 3; the indicator information corresponding to the target chrominance prediction method is generated through the association relationship between the chrominance prediction method and the indicator information The original chrominance information and the predicted chrominance information are subtracted to obtain chrominance residual information; wherein the predicted chrominance information is chrominance prediction by the target chrominance prediction method The chrominance information obtained later; encoding the indication information and the chrominance residual information to obtain a chrominance code stream, and combining the chrominance code stream and the luminance code stream to obtain a video code stream.

In a third aspect, an embodiment of the present application provides an intra-frame chrominance prediction method, which is applied to a video decoder, and the method includes:

Obtain the video code stream output by the video encoder; decode the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and the indication information for determining the chrominance prediction mode; The indication information determines the target chroma prediction mode from at least two chroma prediction modes, the at least two chroma prediction modes including a first type of chroma prediction mode and a second type of chroma prediction mode, the first The chrominance-like prediction method is the intra-frame chrominance prediction method according to any one of claims 1 to 3; according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, through the The target chroma prediction method performs chroma prediction on the chroma components to obtain a chroma prediction result; performs color based on the residual obtained after decoding the chroma residual information in the video bitstream and the chroma prediction result. Reconstruction to get the output chromaticity.

In a fourth aspect, an embodiment of the present application provides an intra-frame chroma prediction method, including:

The video encoder encodes the luminance component to obtain the luminance code stream; obtains the encoded and reconstructed luminance component, the encoded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the chrominance block to be encoded; Determine the target chrominance prediction method with the smallest rate-distortion cost among the two chrominance prediction methods; wherein, the at least two chrominance prediction methods include a first-type chrominance prediction method and a second-type chrominance prediction method. The first type of chrominance prediction method is the intra-frame chrominance prediction method according to any one of claims 1 to 3; the target chrominance prediction method is generated through the association relationship between the chrominance prediction method and the indication information Indication information; subtracting the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is the color of the target chrominance prediction mode Chrominance information obtained after degree prediction; encoding the indication information and the chrominance residual error to obtain a chrominance code stream, and combining the chrominance code stream and the luminance code stream to obtain a video code stream;

The video decoder obtains the video code stream; decodes the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and the indication information; The target chrominance prediction mode is determined in the at least two chrominance prediction modes; according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, the target chrominance prediction mode The chrominance component performs chrominance prediction to obtain the chrominance prediction result; the chrominance reconstruction is performed according to the residual error obtained after decoding the chrominance residual information in the video bitstream and the chrominance prediction result to obtain the output chrominance .

In a fifth aspect, an embodiment of the present application provides a video encoding and decoding system, including a video encoder and a video decoder;

The video encoder is used to encode the luminance component to obtain the luminance code stream; obtain the encoded and reconstructed luminance component, the encoded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the chrominance block to be encoded; pass rate distortion Optimizing the determination of a target chrominance prediction method with the smallest rate-distortion cost from at least two chrominance prediction methods; wherein the at least two chrominance prediction methods include a first-type chrominance prediction method and a second-type chrominance prediction method The first type of chrominance prediction method is the intra-frame chrominance prediction method according to any one of claims 1 to 3; the target chrominance is generated through the association relationship between the chrominance prediction method and the indication information Indication information of the prediction mode; subtracting the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is predicted by the target chrominance The chrominance information obtained after chrominance prediction is performed in a manner; the indication information and the chrominance residual are encoded to obtain a chrominance code stream, and the chrominance code stream and the luminance code stream are combined to obtain a video code stream ；

The video decoder is used to obtain the video code stream; decode the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and the indication information; according to the indication information , Determining the target chrominance prediction mode from the at least two chrominance prediction modes; according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, predict the target chrominance The chrominance component is predicted by the chrominance component to obtain the chrominance prediction result; the chrominance reconstruction is performed according to the residual error obtained after decoding the chrominance residual information in the video bitstream and the chrominance prediction result to obtain Output chromaticity.

In a sixth aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program that is stored in the memory and can run on the processor. The processor executes the computer program when the computer program is executed. The intra-frame chroma prediction method according to any one of the above-mentioned first aspects.

In a seventh aspect, an embodiment of the present application provides a video encoder, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, The intra-frame chrominance prediction method according to any one of the above-mentioned second aspects is implemented.

In an eighth aspect, an embodiment of the present application provides a video decoder, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, The intra-frame chrominance prediction method according to any one of the above-mentioned third aspects is implemented.

In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, implements the first aspect or the second aspect or the first aspect described above. The intra-frame chroma prediction method according to any one of the three aspects.

In a tenth aspect, the embodiments of the present application provide a computer program product. When the computer program product runs on a terminal device or a video encoder or a video decoder, the terminal device or a video encoder or a video decoder can execute the above-mentioned first Aspect or the intra-frame chroma prediction method according to any one of the second aspect or the third aspect.

Beneficial effect

In the embodiment of the application, the chromaticity prediction is performed through the image color sub-network in the chromaticity prediction convolutional neural network model and the corresponding input parameters, that is, the chromaticity prediction problem is modeled as an image coloring problem, which has high universality. In addition, the chroma prediction based on the color sub-network of the image can save the bit rate.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present application. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative labor.

FIG. 1 is a schematic block diagram of the flow of an intra-frame chrominance prediction method provided by an embodiment of the application;

FIG. 2 is a schematic block diagram of the process of reconstructing adjacent chrominance blocks according to an embodiment of the application;

FIG. 3 is a schematic diagram of a reconstruction process of adjacent chrominance blocks provided by an embodiment of this application;

4 is a schematic diagram of an intra-frame chroma prediction method based on a convolutional neural network provided by an embodiment of the application;

FIG. 5 is a schematic block diagram of the structure of an intra-frame chrominance prediction apparatus provided by an embodiment of the application;

6 is a schematic block diagram of the flow of an intra-frame chrominance prediction method provided by an embodiment of the application;

FIG. 7 is a schematic diagram of an encoding process of a video encoder provided by an embodiment of the application;

FIG. 8 is a schematic block diagram of the structure of an intra-frame chrominance prediction apparatus provided by an embodiment of the application;

9 is a schematic block diagram of the flow of an intra-frame chroma prediction method provided by an embodiment of the application;

FIG. 10 is a schematic diagram of a decoding process of a video decoder provided by an embodiment of the application;

FIG. 11 is a schematic block diagram of the structure of an intra-frame chrominance prediction apparatus provided by an embodiment of this application;

FIG. 12 is a schematic block diagram of the structure of a video encoding and decoding system provided by an embodiment of this application;

FIG. 13 is a schematic diagram of interaction between a video encoder and a video decoder provided by an embodiment of the application;

FIG. 14 is a schematic structural diagram of a terminal device provided by an embodiment of the application;

FIG. 15 is a schematic structural diagram of a video encoder provided by an embodiment of the application;

FIG. 16 is a schematic structural diagram of a video decoder provided by an embodiment of the application.

Embodiments of the present invention

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. In order to illustrate the technical solution described in the present application, specific embodiments are used for description below.

Example one

Please refer to FIG. 1, which is a schematic block diagram of the flow of an intra-frame chrominance prediction method provided by an embodiment of this application. The method may include the following steps:

Step 101: Obtain an encoded or decoded and reconstructed luminance component.

It should be noted that the above-mentioned encoded or decoded and reconstructed luminance component can be the luminance component corresponding to any color space and any video format, that is, the intra-frame chrominance prediction method provided by the embodiment of this application can be applied to any color space and any video format. Video format. For example, the above-mentioned encoded or decoded and reconstructed luminance component is the luminance component Y in the YCbCr 4:2:0 format.

Step 102: Down-sampling the encoded or decoded and reconstructed luminance components.

It should be noted that the above-mentioned down-sampling method may be any down-sampling method in the prior art, or brightness down-sampling may be performed through a convolutional neural network.

In some embodiments, the aforementioned chrominance prediction convolutional neural network model may further include a luminance down-sampling sub-network. At this time, the foregoing specific process of down-sampling the encoded or decoded and reconstructed luminance component may include: down-sampling the encoded or decoded and reconstructed luminance component through the luminance down-sampling sub-network.

Wherein, the above-mentioned chrominance prediction convolutional neural network model may include a luminance down-sampling sub-network in addition to the following image color sub-network. Through the brightness down-sampling sub-network, the input coded or decoded and reconstructed brightness components can be down-sampled to obtain the down-sampled coded or decoded and reconstructed brightness components. For example, the 4N×4N encoded or decoded and reconstructed luminance component is input to the luminance down-sampling sub-network, and after the luminance down-sampling sub-network is down-sampled, the 2N×2N encoded or decoded and reconstructed luminance component is output, where , N is 64.

In addition, the output layer of the luminance down-sampling sub-network can include one or more kernel functions, that is, the luminance down-sampling network can output one or more down-sampling results, and one down-sampling result corresponds to a down-sampling encoded or decoded reconstruction The brightness component. Compared with one down-sampling result, chroma prediction through multiple down-sampling results can further improve the chroma prediction performance.

The above-mentioned brightness down-sampling sub-network can be specifically a convolutional neural network. When the color space and video format are in the YCbCr 4:2:0 format, the hyper-parameters and structure of the brightness down-sampling sub-network can be specifically shown in Table 1 below.

Table 1

It should be noted that the structure and hyper-parameters of the brightness down-sampling sub-network shown in Table 1 above are merely illustrative. In specific applications, the hyperparameters and structure in the brightness down-sampling sub-network can be adjusted according to actual needs. For example, when the color space and video format are YCbCr 4:4:4, the second layer stride in the brightness downsampling sub-network is set to 1. When the color space and video format are YCbCr 4:2:2, the brightness is down The sampling sub-network can only be executed in the vertical or horizontal direction.

It is worth noting that, compared with the ordinary down-sampling method, by performing down-sampling through the above-mentioned luminance down-sampling sub-network, more luminance information can be obtained, so as to further improve the performance of subsequent chrominance prediction.

Step 103: Input the preset parameters into the image coloring sub-network in the pre-trained chroma prediction convolutional neural network model to obtain the chroma components output by the image coloring sub-network;

Among them, the preset parameters include down-sampled encoded or decoded and reconstructed luminance components, or down-sampled encoded or decoded and reconstructed luminance components and target parameters. The target parameters include encoding distortion and encoded or decoded At least one of the reconstructed adjacent chrominance blocks is decoded.

It should be noted that the above-mentioned preset parameters may only include down-sampled encoded or decoded and reconstructed luminance components, may include down-sampled encoded or decoded and reconstructed luminance components and encoding distortion, and may include down-sampling. The subsequent coded or decoded and reconstructed luminance component and adjacent chrominance blocks may also include the down-sampled coded or decoded and reconstructed luminance component, coding distortion, and the coded or decoded and reconstructed adjacent chrominance Piece.

Among them, the adjacent chrominance blocks that have been coded or decoded and reconstructed can improve the chroma prediction performance of the image coloring network and the training speed of the network model, and the coding distortion can eliminate the negative impact of compression distortion. Based on this, when the preset parameters include down-sampled coded or decoded and reconstructed luminance components, coding distortion, and adjacent chroma blocks, the performance of the image coloring network is the best, that is, the intra-frame chroma prediction performance Preferably; when the preset parameters include the down-sampling coded or decoded and reconstructed luminance component and coding distortion, or the down-sampled coded or decoded reconstructed luminance component and the coded or decoded reconstructed phase When adjacent to chrominance blocks, the performance of the image coloring network is second; when the preset parameters only include down-sampled encoded or decoded and reconstructed luminance components, the performance of the image coloring network is the worst, that is, the intra-frame color Degree prediction performance is the worst.

It should be understood that even if the preset parameters only include the down-sampled encoded or decoded and reconstructed luminance components, the color prediction results can still be obtained through the image coloring network, that is, the objectives of the embodiments of the present application can still be achieved.

It is worth noting that the aforementioned encoding distortion degree can be specifically expressed as an image block characterized by a quantization parameter. Wherein, the value of the encoding distortion degree can be any value from 0 to 51. For example, the coding distortion degree is 10, and the coding distortion degree is specifically a 2N×2N image block, and the value of each pixel in the image block is 10.

The aforementioned adjacent chrominance block refers to an image block that includes adjacent chrominance information, and the adjacent chrominance block is reconstructed in advance. In some embodiments, when the above-mentioned preset parameters include adjacent chrominance blocks that have been encoded or decoded and reconstructed, referring to the schematic block diagram of the flow of the adjacent chrominance block reconstruction process shown in FIG. 2, the above-mentioned intra-frame chrominance Forecasting methods can also include:

Step 201: Cut out the target brightness component block from the coded or decoded and reconstructed brightness component.

It should be noted that the aforementioned target luminance component block generally refers to the luminance component block located at the lower right of the encoded or decoded and reconstructed luminance component. For example, the coded or decoded and reconstructed luminance component is a 4N×4N luminance block, and the 4N×4N luminance component block is divided into 4 2N×2N luminance component blocks, and the 2N×2N luminance component block at the bottom right is Target luminance component block.

Step 202: Perform chroma prediction on the target luminance component block by using a preset chroma prediction mode to obtain a predicted chroma.

It should be noted that the above-mentioned preset chromaticity prediction method can be specifically any chromaticity prediction method in the prior art, for example, a linear prediction model CCLM or a multi-directional linear model MDLM. The traditional linear chrominance prediction model is used to predict the chrominance of the target luminance component block to obtain the predicted chrominance Cb and Cr.

Step 203: Use the predicted chrominance as the initial chrominance information of the chrominance block to be predicted.

In order to better introduce the foregoing adjacent chrominance block reconstruction process, the following will be introduced in conjunction with the schematic diagram of the adjacent chrominance block reconstruction process shown in FIG. 3.

As shown in Fig. 3, the size of the coded or decoded and reconstructed brightness component 31 is 4N×4N, and the brightness component includes brightness component blocks of size 2N×2N and

codes

1, 2, 3, and 4 respectively, where, Luminance component block 1 is located at the upper left, luminance component block 2 is located at the upper right, luminance component block 3 is located at the lower left, and luminance component block 4 is located at the lower right. The luminance component block 32 can be obtained by cropping, and then the 2N×2N luminance component block 32 is input into the linear prediction model CCLM to obtain the predicted chrominance Cb and Cr 34, and then the N×N chrominance block Cb and Cr are respectively filled in The vacant parts to the corresponding chrominance block 35, that is, the chrominance blocks Cb and Cr are respectively filled to the position of the question mark in FIG. 3, as the initial chrominance information of the 2N×2N chrominance block to be predicted.

It should be noted that the input of the above-mentioned image coloring sub-network is a grayscale image, and the output is a corresponding color image. In the embodiment of the present application, the chromaticity prediction problem is modeled as an image coloring problem, that is, the purpose of intra-frame chromaticity prediction is achieved through image coloring.

As an example and not a limitation, the structure and hyperparameters of the above-mentioned image color sub-network may be shown in Table 2 below.

Table 2

It should be understood that the structure and hyperparameters of the image color sub-network shown in Table 2 above are only an example. In specific applications, the hyperparameters and structures in the above-mentioned image coloring sub-network can be adjusted as needed.

The above-mentioned chrominance prediction convolutional neural network model includes the above-mentioned image coloring sub-network. In some embodiments, it may also include a luminance down-sampling sub-network. The chrominance prediction convolutional neural network model is pre-trained.

When the chroma prediction convolutional neural network model includes a luminance down-sampling sub-network and an image color sub-network, during the training process of the chroma convolutional neural network model, the loss function is specifically:

L ₂ =λ||Cb'-Cb|| ² +(1-λ)||Cr'-Cr|| ² , where λ is the weight, Cb' and Cr' are cropped from the output of the image coloring network The size of the obtained chrominance component is N×N. Cb and Cr are the true values of chrominance components with a size of N×N.

Among them, Cb', Cr'=F ₂ (F ₁ (Y), D, Cb, Cr), F ₂ is the image coloring network, F ₁ (Y) is the down-sampled coded or decoded reconstructed brightness Component, the size is 2N×2N; D is the coding distortion degree, and the size is 2N×2N; Cb, Cr are adjacent chrominance information in adjacent chrominance blocks, and the adjacent chrominance block size is 2N×2N. The batch size and learning rate during training are set to 128 and 1×10 ^{-4 respectively} . λ can be set to 0.5. The training sample data set may include 886 images from the UCID database and 400 images from the DIV2K database.

Step 104: Cut out the target chrominance component block from the chrominance component, and the target chrominance component block is the final chrominance prediction result.

Specifically, after the preset parameters are input to the image coloring sub-network, the image coloring sub-network will output the corresponding chroma components, and then the corresponding chroma component blocks are cut out from the output of the image coloring sub-network, To get the predicted chromaticity. That is, in some embodiments, the above-mentioned specific process of obtaining the color prediction result according to the chroma component may include: cropping the target chroma component block from the chroma component, and the target chroma component block is the coded or decoded reconstructed luminance. The chroma prediction result corresponding to the component.

For example, when the size of the chrominance component output by the color sub-network on the image is 2N×2N, an N×N target chrominance component block is cropped from the 2N×2N chrominance component, and the target luminance component block is 2N×2N The chroma block in the lower right of the chroma component.

In order to better introduce the intra-frame chrominance prediction method provided by the embodiments of the present application, the following will be introduced in conjunction with the schematic diagram of the intra-frame chrominance prediction method based on the convolutional neural network shown in FIG. 4.

As shown in Figure 4, the chroma prediction convolutional neural network model includes a luminance down-sampling sub-network 41 and an image color sub-network 42. The size of the encoded or decoded and reconstructed luminance component 43 is 4N×4N, including four 2N ×2N brightness component blocks, the 4 2N×2N brightness component blocks are numbered 1, 2, 3, and 4 respectively. Cut out the 2N×2N luminance component block 4 from the encoded or decoded and reconstructed luminance component 43 as the target luminance component block, and input the target luminance component block to the linear chrominance prediction model CCLM to obtain the linear chrominance prediction model CCLM The output results Cb and Cr are filled in the vacant parts in the adjacent chrominance block as the initial chrominance components of the chrominance block to be predicted.

The encoded or decoded and reconstructed luminance component 43 is input to the luminance down-sampling network 41 to obtain a plurality of down-sampled encoded or decoded and reconstructed luminance components. Then multiple 2N×2N down-sampled encoded or decoded and reconstructed luminance components 44, reconstructed 2N×2N adjacent chrominance blocks 45, and 2N×2N encoding distortion 46 are input to the image The color sub-network 42, the color sub-network on the image outputs two 2N×2N chrominance components 47, respectively cropping out N×N Cb′ and Cr′ from the two 2N×2N chrominance components, and the cropped N ×N Cb' and Cr' are the final chromaticity prediction results.

Correspondingly, referring to the schematic structural block diagram of an intra-frame chrominance prediction apparatus shown in FIG. 5, the apparatus may include:

The luminance component acquisition module 51 is configured to acquire the encoded or decoded and reconstructed luminance component;

The down-sampling module 52 is used for down-sampling the encoded or decoded and reconstructed luminance components;

The coloring module 53 is used to input preset parameters into the image coloring sub-network in the pre-trained chroma prediction convolutional neural network model to obtain the chroma components output by the image coloring sub-network; wherein, the preset parameters include The coded or decoded and reconstructed luminance component after downsampling, or the coded or decoded and reconstructed luminance component after downsampling and the target parameter. The target parameter includes the coding distortion and the adjacent chrominance that has been coded or decoded and reconstructed At least one of the blocks;

The prediction module 54 is used to cut out the target chrominance component block from the chrominance component, and the target chrominance component block is the final chrominance prediction result.

In some embodiments, when the preset parameter includes adjacent chrominance blocks, the apparatus may further include:

The cropping module is used to crop the target brightness component block from the coded or decoded and reconstructed brightness component;

The chrominance prediction module is used to predict the chrominance of the target luminance component block through a preset chrominance prediction method to obtain the predicted chrominance;

The reconstruction module uses the predicted chrominance as the initial chrominance component of the chrominance block to be predicted.

In some embodiments, the above-mentioned chrominance prediction convolutional neural network model further includes a luminance down-sampling sub-network; the above-mentioned down-sampling module is specifically used to: use the luminance down-sampling sub-network to download the encoded or decoded and reconstructed luminance components sampling.

The embodiment of the present application also provides another preferred embodiment of the intra-frame chrominance prediction device. In this embodiment, the intra-frame chrominance prediction device includes a processor, wherein the following program modules used to execute the memory are processed: : Luminance component acquisition module, used to obtain the encoded or decoded and reconstructed luminance components; down-sampling module, used to down-sample the encoded or decoded and reconstructed luminance components; coloring module, used to input preset parameters To the image color sub-network in the pre-trained chroma prediction convolutional neural network model, the chroma components output by the image color sub-network are obtained; among them, the preset parameters include the down-sampled coded or decoded reconstructed brightness Component, or includes down-sampled encoded or decoded and reconstructed luminance component and target parameter, the target parameter includes at least one of encoding distortion and encoded or decoded and reconstructed adjacent chrominance blocks; a prediction module for The target chrominance component block is cut out from the chrominance component, and the target chrominance component block is the final chrominance prediction result.

It should be noted that the foregoing intra-frame chrominance prediction device corresponds to the foregoing intra-frame chrominance prediction method one-to-one. For related introduction, please refer to the corresponding content above, which will not be repeated here.

It can be seen that the intra-frame chromaticity prediction scheme based on the convolutional neural network provided by the embodiment of the present application performs chromaticity prediction through the image color sub-network and the corresponding input parameters, so as to model the chromaticity prediction problem as an image The color problem is more universal. In addition, the chroma prediction based on the color sub-network of the image can save the bit rate. It can be obtained through experiments that the intra-frame chrominance prediction scheme based on the convolutional neural network provided by the embodiment of the present application can save 4.235% of the code rate on average compared with the existing chrominance prediction method.

Example two

The intra-frame chroma prediction scheme based on the convolutional neural network provided by the embodiments of the present application can be applied to the video encoding and decoding process. In order to further improve the performance of video coding and decoding, the rate-distortion cost competition can be carried out between the intra-frame chroma prediction method based on convolutional neural network and the traditional chroma prediction method, and the chroma prediction method with the smallest rate-distortion cost can be selected for video Codec. This embodiment will introduce the chroma coding process.

Refer to FIG. 6, which is a schematic block diagram of the flow of an intra-frame chrominance prediction method provided by an embodiment of this application. The method may be specifically applied to a video encoder. The method may include the following steps:

Step 601: Encode the luminance component to obtain a luminance code stream;

Step 602: Obtain the coded and reconstructed luminance component, the coded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the to-be-coded chrominance block.

It can be understood that the above-mentioned coded or decoded and reconstructed luminance component Y, chrominance components Cb, Cr, and adjacent chrominance information can all be included in the coding block. Wherein, the aforementioned adjacent chrominance information is specifically represented as adjacent chrominance blocks.

Step 603: Determine the target chrominance prediction method with the smallest rate-distortion cost from at least two chrominance prediction methods through rate-distortion optimization; wherein, the at least two chrominance prediction methods include the first type of chrominance prediction method and the second type. For the chroma prediction method, the first type of chroma prediction method is the intra-frame chroma prediction method as in any one of the foregoing first embodiment.

It should be noted that the video encoder includes at least two chroma prediction methods, and the at least two chroma prediction methods include a first type of chroma prediction method and a second type of chroma prediction method. The first type of chroma prediction method refers to the intra-frame chroma prediction method based on the convolutional neural network provided by the embodiment of the application, and the second type of chroma prediction method may refer to the traditional intra-frame chroma prediction method, and the traditional Intra-frame chroma prediction methods include angle prediction, linear model CCLM, multi-directional linear model MDLM, and so on. It should be understood that the foregoing second-type chrominance prediction method may include one or more traditional intra-frame chrominance prediction methods.

Through rate-distortion optimization, the chroma prediction method with the least rate-distortion cost can be determined from a variety of chroma prediction methods. In some embodiments, the above-mentioned specific process of determining the target chrominance prediction method with the smallest rate-distortion cost from at least two chrominance prediction methods through rate-distortion optimization may include: separately calculating the rates corresponding to the at least two chrominance prediction methods. Distortion cost: Determine the chromaticity prediction method with the least cost-distortion cost as the target chromaticity prediction method.

Step 604: Generate the indication information corresponding to the target chromaticity prediction mode through the association relationship between the chromaticity prediction mode and the indication information.

It should be noted that the above-mentioned association relationship is established in advance, and the indication information corresponding to each chromaticity prediction method can be determined through the association relationship. For example, when the indication information is a binary flag bit value, the mapping relationship between each chroma prediction method and the corresponding value is established in advance, and the specific expression is: the flag bit value corresponding to the first chroma prediction method is 00, The value of the number of flag bits corresponding to the second chromaticity prediction method is 01, and so on.

After obtaining the indication information corresponding to the chroma prediction mode through the association relationship, the corresponding indication information can be generated according to the content of the corresponding indication information. For example, when the value corresponding to the target chromaticity prediction mode is 1, the value of the binary flag bit is set to 1 to generate the indication information corresponding to the target chromaticity prediction mode. The indication information is used to indicate which chrominance prediction mode is selected for chrominance prediction.

In some embodiments, the above-mentioned indication information is specifically a flag bit value. The above-mentioned specific process of generating the indication information corresponding to the target chromaticity prediction mode through the association relationship between the chrominance prediction mode and the indication information may include: setting the flag bit through the association relationship between the chrominance prediction mode and the value of the flag bit Is the corresponding value to obtain the indication information corresponding to the target chromaticity prediction mode.

In this embodiment, the binary flag bit value corresponding to the intra-frame chroma prediction method based on the convolutional neural network in the first embodiment can be set to 1, and the binary flag bit value corresponding to the second type of chroma prediction method can be set Is 0. At this time, if it is determined through the rate-distortion optimization that the chroma prediction method based on the convolutional neural network has the smallest rate-distortion cost, then the value of the binary flag bit is set to 1. On the contrary, if the smallest rate-distortion cost is the first For the second type of chroma prediction method, the value of the binary flag bit is set to 0. It should be understood that when the second type of chrominance prediction method includes multiple traditional chrominance prediction methods, two or three binary flags can be used to represent the corresponding chrominance prediction method, for example, two binary flags can be used to represent the corresponding chrominance prediction method. For the corresponding chrominance prediction method, the value of the binary flag corresponding to the first traditional chrominance prediction method is 00, the value of the binary flag corresponding to the second traditional chrominance prediction method is 01, and so on.

Step 605: Perform a subtraction operation on the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is obtained after chrominance prediction is performed through the target chrominance prediction method Chromaticity information.

Wherein, the chrominance information obtained by the above prediction is information obtained by performing chrominance prediction on the chrominance block to be predicted by executing the determined target chrominance prediction method, and the specific process is not repeated here.

Step 606: Encode the indication information and the chrominance residual information to obtain a chrominance code stream, which is combined with the luminance code stream to obtain a video code stream.

Specifically, lossless true coding is performed on the indication information, and corresponding residual coding is performed on the chrominance residual information to obtain the output code stream of the video encoder.

In order to better introduce the encoding process of the video encoder provided in this embodiment, the following will describe with reference to the schematic diagram of the encoding process of the video encoder shown in FIG. 7.

As shown in Figure 7, the luminance component is coded to obtain the luminance code stream; the chrominance coding block to be predicted 71 is input to the video encoder 72, and the chrominance coding block to be predicted includes the coded and reconstructed luminance component and the coded reconstruction. The adjacent chromaticity information and other information. The video encoder executes the traditional intra-frame chrominance prediction method and the intra-frame chrominance prediction method based on convolutional neural network respectively. Through rate-distortion optimization, the rate-distortion cost value of each chrominance prediction method is calculated, and then the rate-distortion code is compared. For the value of the value, select the chroma prediction method corresponding to the minimum rate-distortion cost value as the target chroma prediction method; then based on the target chroma prediction method, set the binary flag bit to the corresponding value, and then encode the binary flag bit and predict it The chroma coding block performs residual coding to obtain the chroma code stream 73. The chrominance code stream and the luminance code stream are combined into a video code stream, which is sent to the video decoder for corresponding decoding process.

Correspondingly, referring to the schematic structural block diagram of an intra-frame chrominance prediction apparatus shown in FIG. 8, the apparatus may include:

The luminance encoding module 81 is used to encode the luminance component to obtain a luminance code stream;

The obtaining module 82 is configured to obtain the coded and reconstructed luminance component, the coded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the to-be-coded chrominance block;

The second determining module 83 is configured to determine the target chrominance prediction method with the minimum rate-distortion cost from at least two chrominance prediction methods through rate-distortion optimization; wherein, the at least two chrominance prediction methods include the first type of chrominance prediction Method and the second type of chroma prediction method, the first type of chroma prediction method is the intra-frame chroma prediction method as in any one of the foregoing embodiment;

The generating module 84 is configured to generate the indication information corresponding to the target chromaticity prediction mode through the association relationship between the chromaticity prediction mode and the indication information;

The subtraction module 85 is used to perform a subtraction operation between the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is the chrominance prediction through the target chrominance prediction method Chromaticity information obtained afterwards;

The encoding module 86 is configured to encode the indication information and the chrominance residual information to obtain a chrominance code stream, and combine the chrominance code stream and the luminance code stream to obtain a video code stream.

In some embodiments, the above-mentioned second determining module is specifically configured to: respectively calculate the rate-distortion cost values corresponding to at least two chrominance prediction modes; and determine the chrominance prediction mode with the smallest rate-distortion cost value as the target chrominance prediction mode.

In some embodiments, the above-mentioned indication information is specifically a flag bit value. The above-mentioned generating module is specifically used to set the flag bit to a corresponding value through the association relationship between the chrominance prediction mode and the value of the flag bit, so as to obtain the indication information corresponding to the target chrominance prediction mode.

The embodiment of the present application also provides another preferred embodiment of the intra-frame chrominance prediction device. In this embodiment, the intra-frame chrominance prediction device includes a processor, wherein the following program modules used to execute the memory are processed: : Luminance encoding module, used to encode the luminance component to obtain the luminance code stream; acquisition module, used to obtain the encoded and reconstructed luminance component, the encoded and reconstructed adjacent chrominance information, and the original color corresponding to the chrominance block to be encoded Degree information; a second determination module for determining the target chrominance prediction method with the least rate-distortion cost from at least two chrominance prediction methods through rate-distortion optimization; wherein the at least two chrominance prediction methods include the first type of color The first type of chroma prediction method is the intra-frame chroma prediction method as in any one of the above-mentioned embodiment; the generation module is used to pass the chroma prediction method and the indication information. The correlation relationship of the target chrominance prediction method is generated; the subtraction module is used to subtract the original chrominance information and the predicted chrominance information to obtain the chrominance residual information; among them, the predicted chrominance information The chrominance information is the chrominance information obtained after the chrominance prediction is performed by the target chrominance prediction method; the coding module is used to encode the indication information and the chrominance residual information to obtain the chrominance code stream, and the chrominance code The stream and the luminance code stream are combined to obtain the video code stream.

It can be seen that the rate-distortion cost competition is performed between the traditional intra-frame chrominance prediction method and the intra-frame chrominance prediction method based on the convolutional neural network provided in the embodiment of the application, and an increase is used to indicate which chrominance is selected. The indication information of the prediction mode can further improve the chroma coding performance.

Example three

After introducing the video encoding process, this embodiment introduces the video decoding process. The video decoding process in this embodiment corresponds to the video encoding process in the second embodiment above.

Refer to FIG. 9, which is a schematic block diagram of the flow of an intra-frame chrominance prediction method provided by an embodiment of this application. The method may be applied to a video decoder. The method may include the following steps:

Step 901: Obtain a code stream output by the video encoder.

Step 902: Decode the video code stream to obtain decoded and reconstructed luminance components, decoded and reconstructed adjacent chrominance information, and indication information for determining a chrominance prediction mode.

Specifically, the video decoder receives the video code stream output by the video encoder, and then decodes the code stream to obtain corresponding information. Wherein, the above-mentioned indication information may specifically be a binary flag bit.

Step 903: According to the instruction information, determine the target chroma prediction mode from at least two chroma prediction modes, the at least two chroma prediction modes include the first type of chroma prediction mode and the second type of chroma prediction mode, the first type The chroma prediction method is the intra-frame chroma prediction method as in any one of the above-mentioned first embodiment.

Specifically, after the instruction information is obtained by decoding, the selected target chromaticity prediction mode can be determined according to the instruction information. For example, when the indication information is specifically the value of the flag bit; the specific process of determining the target chroma prediction mode from at least two chroma prediction modes according to the indication information may include: when the bit value of the flag is the first value, the first The second-type chromaticity prediction method is determined as the target chromaticity prediction method; when the flag bit value is the second value, the second-type chromaticity prediction method is determined as the target chromaticity prediction method. Wherein, the above-mentioned first value may be 1, and correspondingly, the second value is 0; the first value may also be 0, and correspondingly, the second value is 1.

Step 904: According to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, perform chrominance prediction on the chrominance component through the target chrominance prediction mode to obtain a chrominance prediction result.

It is understandable that after the target chromaticity prediction mode is selected according to the instruction information, the target chromaticity prediction mode can be executed to perform chromaticity prediction, so as to obtain the corresponding chromaticity prediction result. Wherein, if the target chromaticity prediction method is the intra-frame chromaticity prediction method based on the convolutional neural network in the first embodiment, the specific process of chromaticity prediction can be referred to the corresponding content above, which will not be repeated here.

Step 905: Perform chroma reconstruction according to the residual and chroma prediction result obtained after decoding the chroma residual information in the bitstream to obtain the output chroma.

In order to better introduce the video decoding process, the following will be introduced in conjunction with the schematic diagram of the decoding process of the video decoder shown in FIG. 10.

As shown in Figure 10, the video decoder 101 receives the input code stream 102, first decodes the luminance component, and then decodes the binary flag bit to obtain the binary flag bit. According to the binary flag bit, the traditional intra-frame chrominance prediction method is selected. For chroma prediction, choose the intra-frame chroma prediction method based on convolutional neural network for chroma prediction; then execute the selected target chroma prediction method to perform chroma prediction to obtain the chroma prediction result; based on the obtained chroma prediction The result and the residual decoding result are subjected to chroma reconstruction, and the output chroma 103 is obtained.

Correspondingly, referring to the schematic structural block diagram of an intra-frame chrominance prediction apparatus shown in FIG. 11, the apparatus may include:

The code stream obtaining module 111 is used to obtain the code stream output by the video encoder;

The decoding module 112 is configured to decode the video code stream to obtain decoded and reconstructed luminance components, decoded and reconstructed adjacent chrominance information, and indication information for determining a chrominance prediction mode;

The first determining module 113 is configured to determine the target chrominance prediction mode from at least two chrominance prediction modes according to the instruction information. The at least two chrominance prediction modes include the first type of chrominance prediction mode and the second type of chrominance prediction mode. Method, the first type of chroma prediction method is the intra-frame chroma prediction method as in any one of the foregoing embodiment;

The chroma prediction module 114 is configured to perform chroma prediction on the chroma component by the target chroma prediction mode according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chroma information to obtain a chroma prediction result;

The chrominance reconstruction module 115 is configured to perform chrominance reconstruction according to the residual and chrominance prediction result obtained after decoding the chrominance residual information in the video bitstream to obtain the output chrominance.

In some embodiments, the aforementioned indication information is specifically a flag bit value; the aforementioned first determining module is specifically configured to: when the flag bit value is the first value, determine the first type of chromaticity prediction mode as the target chromaticity prediction mode; When the value of the number of flag bits is the second value, the second type of chromaticity prediction mode is determined as the target chromaticity prediction mode.

The embodiment of the present application also provides another preferred embodiment of the intra-frame chrominance prediction device. In this embodiment, the intra-frame chrominance prediction device includes a processor, wherein the following program modules used to execute the memory are processed: : Code stream acquisition module, used to obtain the code stream output by the video encoder; decoding module, used to decode the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and to determine Indication information of the chrominance prediction mode; a first determining module, configured to determine the target chrominance prediction mode from at least two chrominance prediction modes according to the indication information, the at least two chrominance prediction modes including the first type of chrominance prediction mode And the second type of chroma prediction method, the first type of chroma prediction method is the intra-frame chroma prediction method as in any one of the above embodiment; the chroma prediction module is used to reconstruct the decoded luminance component and the decoded reconstruction The adjacent chrominance information of the chrominance component is predicted by the target chrominance prediction method to obtain the chrominance prediction result; the chrominance reconstruction module is used to decode the chrominance residual information in the video stream The obtained residual and chromaticity prediction results are subjected to chromaticity reconstruction to obtain the output chromaticity.

It should be noted that the foregoing intra-frame chrominance prediction apparatus corresponds to the intra-frame chrominance prediction method in the foregoing embodiment one-to-one. For related introduction, please refer to the corresponding content above, which will not be repeated here.

Example four

Refer to FIG. 12, which is a schematic block diagram of a structure of a video encoding and decoding system provided by an embodiment of this application. The system may include a video encoder 121 and a video decoder 122. Of course, the system also includes an encoding transmission sub-system 123 for transmitting the code stream, which is between the video encoder and the video decoder, and is used to transmit the code stream output by the video encoder to the video decoder. .

The working flow and interaction flow of the video encoder and the video decoder can be seen in Figure 13 below, which will not be repeated here.

It should be noted that for the intra-frame chroma prediction method based on the convolutional neural network, the encoding process of the video encoder, and the decoding process of the video decoder, please refer to the corresponding content above, which will not be repeated here.

Correspondingly, referring to the schematic diagram of interaction between the video encoder and the video decoder shown in FIG. 13, the interaction process of the intra-frame chrominance prediction system may include the following steps:

Step 1301: The video encoder encodes the luminance component to obtain a luminance code stream.

Step 1302: Obtain the coded and reconstructed luminance component and the coded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the chrominance block to be coded.

Step 1303: The video encoder determines the target chrominance prediction method with the smallest rate-distortion cost from at least two chrominance prediction methods through rate-distortion optimization; wherein, the at least two chrominance prediction methods include the first type of chrominance prediction method and The second type of chroma prediction method, the first type of chroma prediction method is the intra-frame chroma prediction method according to any one of the above-mentioned first aspects.

Step 1304: The video encoder generates the indication information of the target chrominance prediction mode through the association relationship between the chrominance prediction mode and the indication information.

Step 1305: The video encoder performs a subtraction operation on the original chrominance information and the predicted chrominance information to obtain chrominance residual information.

Step 1306: The video encoder encodes the indication information and the chrominance residual information to obtain a chrominance code stream, which is combined with the luminance code stream to obtain a video code stream.

Step 1307: The video decoder obtains the video code stream output by the video encoder.

Step 1308: The video decoder decodes the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and the indication information.

Step 1309: The video decoder determines the target chrominance prediction mode from at least two chrominance prediction modes according to the instruction information.

Step 1310: According to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, the video decoder performs chrominance prediction on the chrominance component through the target chrominance prediction mode to obtain a chrominance prediction result.

Step 1311. The video decoder performs chroma reconstruction based on the residual and chroma prediction result obtained after decoding the chroma residual information in the bitstream, to obtain the output chroma.

It should be noted that the interaction process between the video encoder and the video decoder is the same as or similar to the above embodiments, please refer to the corresponding content above, which will not be repeated here.

It should be noted that the information exchange and execution process between the above-mentioned devices and units are based on the same concept as the method embodiment of this application, and its specific functions and technical effects can be found in the method embodiment section for details. I won't repeat it here.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

Example five

FIG. 14 is a schematic structural diagram of a terminal device provided by an embodiment of this application. As shown in FIG. 14, the terminal device 14 of this embodiment includes: at least one processor 140, a memory 141, and a computer program 142 that is stored in the memory 141 and can run on the at least one processor 140. When the processor 140 executes the computer program 142, the steps in any embodiment of the intra-frame chrominance prediction method in the first embodiment are implemented.

The terminal device 14 may be a computing device such as a desktop computer, a notebook, or a palmtop computer. The terminal device may include, but is not limited to, a processor 140 and a memory 141. Those skilled in the art can understand that FIG. 14 is only an example of the terminal device 14 and does not constitute a limitation on the terminal device 14. It may include more or less components than shown in the figure, or a combination of certain components, or different components. , For example, can also include input and output devices, network access devices, and so on.

The so-called processor 140 may be a central processing unit (Central Processing Unit, CPU), and the processor 140 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 141 may be an internal storage unit of the terminal device 14 in some embodiments, such as a hard disk or a memory of the terminal device 14. In other embodiments, the memory 141 may also be an external storage device of the terminal device 14, such as a plug-in hard disk equipped on the terminal device 14, a smart media card (SMC), a secure digital (Secure Digital, SD) card, flash card (Flash Card), etc. Further, the memory 141 may also include both an internal storage unit of the terminal device 14 and an external storage device. The memory 141 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 141 can also be used to temporarily store data that has been output or will be output.

FIG. 15 is a schematic structural diagram of a video encoder provided by an embodiment of this application. As shown in FIG. 15, the video encoder 15 of this embodiment includes: at least one processor 150, a memory 151, and a computer program 152 that is stored in the memory 151 and can run on the at least one processor 150, so When the processor 150 executes the computer program 152, the steps in any embodiment of the intra-frame chrominance prediction method in the second embodiment are implemented.

The video encoder may include, but is not limited to, a processor 150 and a memory 151. Those skilled in the art can understand that FIG. 15 is only an example of the video encoder 15 and does not constitute a limitation on the video encoder 15. It may include more or less components than shown in the figure, or a combination of certain components, or different components. The components of, for example, can also include input and output devices, network access devices, and so on.

The so-called processor 150 may be a central processing unit (Central Processing Unit, CPU), and the processor 150 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 151 may be an internal storage unit of the video encoder 15 in some embodiments, such as a hard disk or a memory of the video encoder 15. In other embodiments, the memory 151 may also be an external storage device of the video encoder 15, such as a plug-in hard disk or a smart media card (SMC) equipped on the video encoder 15, Secure Digital (SD) card, Flash Card, etc. Further, the memory 151 may also include both an internal storage unit of the video encoder 15 and an external storage device. The memory 151 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 151 can also be used to temporarily store data that has been output or will be output.

FIG. 16 is a schematic structural diagram of a video decoder provided by an embodiment of this application. As shown in FIG. 16, the video decoder 16 of this embodiment includes: at least one processor 160, a memory 161, and a computer program 162 that is stored in the memory 161 and can run on the at least one processor 160, so When the processor 160 executes the computer program 162, the steps in the embodiment of any intra-frame chrominance prediction method in the third embodiment are implemented.

The video decoder may include, but is not limited to, a processor 160 and a memory 161. Those skilled in the art can understand that FIG. 16 is only an example of the video decoder 16 and does not constitute a limitation on the video decoder 16. It may include more or less components than those shown in the figure, or combine certain components, or be different. The components of, for example, can also include input and output devices, network access devices, and so on.

The so-called processor 160 may be a central processing unit (Central Processing Unit, CPU), and the processor 160 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 161 may be an internal storage unit of the video decoder 16 in some embodiments, such as a hard disk or a memory of the video decoder 16. In other embodiments, the memory 161 may also be an external storage device of the video decoder 16, for example, a plug-in hard disk or a smart memory card (Smart Media Card, SMC) equipped on the video decoder 16, Secure Digital (SD) card, Flash Card, etc. Further, the memory 161 may also include both an internal storage unit of the video decoder 16 and an external storage device. The memory 161 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 161 can also be used to temporarily store data that has been output or will be output.

The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. Intra-frame chroma prediction method.

The embodiments of the present application also provide a computer program product. When the computer program product runs on a terminal device or a video encoder or a video decoder, the terminal device or a video encoder or a video decoder will correspondingly execute the above-mentioned embodiment one or The intra-frame chroma prediction method of any one of Embodiment 2 or Embodiment 3.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

An intra-frame chrominance prediction method, characterized in that it comprises:

Obtain the coded or decoded and reconstructed luminance component;

Down-sampling the encoded or decoded and reconstructed luminance component;

The preset parameters are input to the image coloring sub-network in the pre-trained chroma prediction convolutional neural network model to obtain the chroma components output by the image coloring sub-network; wherein, the preset parameters include down-sampling The coded or decoded and reconstructed luminance component, or the coded or decoded and reconstructed luminance component after downsampling and the target parameter, the target parameter including the coding distortion and the coded or decoded and reconstructed adjacent chrominance At least one of the blocks;

The target chrominance component block is cut out from the chrominance component, and the target chrominance component block is the final chrominance prediction result.
The intra-frame chrominance prediction method according to claim 1, wherein when the preset parameter includes the coded or decoded and reconstructed adjacent chrominance block, the method further comprises:

Crop a target luminance component block from the encoded or decoded and reconstructed luminance component;

Performing chrominance prediction on the target luminance component block by using a preset chrominance prediction mode to obtain the predicted chrominance;

The predicted chrominance is used as the initial chrominance component of the chrominance block to be predicted.
The intra-frame chrominance prediction method according to claim 1, wherein the chrominance prediction convolutional neural network model further comprises a luminance down-sampling sub-network;

The down-sampling of the coded or decoded and reconstructed luminance component includes:

Through the brightness down-sampling sub-network, down-sampling the coded or decoded and reconstructed brightness components.
An intra-frame chrominance prediction method, characterized in that it is applied to a video encoder, and the method includes:

Encode the luminance component to obtain the luminance code stream;

Acquiring the coded and reconstructed luminance component, the coded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the chrominance block to be coded;

Through rate-distortion optimization, the target chrominance prediction method with the least rate-distortion cost is determined from at least two chrominance prediction methods; wherein, the at least two chrominance prediction methods include a first-type chrominance prediction method and a second-type chrominance prediction method. Degree prediction mode, the first type of chroma prediction mode is the intra-frame chroma prediction method according to any one of claims 1 to 3;

Generating the indication information corresponding to the target chromaticity prediction mode through the association relationship between the chromaticity prediction mode and the indication information;

Perform a subtraction operation on the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is after chrominance prediction is performed through the target chrominance prediction method Chromaticity information obtained;

The indication information and the chrominance residual information are encoded to obtain a chrominance code stream, and the chrominance code stream and the luminance code stream are combined to obtain a video code stream.
The intra-frame chrominance prediction method according to claim 4, wherein the determining the target chrominance prediction method with the least rate-distortion cost from at least two chrominance prediction methods through rate-distortion optimization comprises:

Respectively calculating the rate-distortion cost values corresponding to the at least two chromaticity prediction methods;

The chromaticity prediction mode with the least cost-distortion cost is determined as the target chromaticity prediction mode.
The intra-frame chrominance prediction method according to claim 4, wherein the indication information is specifically a flag bit value;

The generating the indication information corresponding to the target chrominance prediction mode through the association relationship between the chrominance prediction mode and the indication information includes:

According to the association relationship between the chromaticity prediction mode and the value of the flag bit, the flag bit is set to the corresponding value to obtain the indication information corresponding to the target chromaticity prediction mode.
An intra-frame chrominance prediction method, characterized in that it is applied to a video decoder, and the method includes:

Obtain the video code stream output by the video encoder;

Decoding the video code stream to obtain decoded and reconstructed luminance components, decoded and reconstructed adjacent chrominance information, and indication information for determining a chrominance prediction mode;

According to the instruction information, a target chroma prediction mode is determined from at least two chroma prediction modes, the at least two chroma prediction modes include a first type of chroma prediction mode and a second type of chroma prediction mode, the The first type of chroma prediction method is the intra-frame chroma prediction method according to any one of claims 1 to 3;

Performing chrominance prediction on the chrominance component through the target chrominance prediction mode according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information to obtain a chrominance prediction result;

Perform chroma reconstruction according to the residual obtained after decoding the chroma residual information in the video bitstream and the chroma prediction result to obtain the output chroma.
The intra-frame chrominance prediction method according to claim 7, wherein the indication information is specifically a flag bit value;

The determining the target chromaticity prediction mode from at least two chromaticity prediction modes according to the instruction information includes:

When the value of the number of flag bits is the first value, determining the first-type chromaticity prediction mode as the target chromaticity prediction mode;

When the value of the number of flag bits is the second value, the second-type chromaticity prediction mode is determined as the target chromaticity prediction mode.
An intra-frame chrominance prediction method, characterized in that it comprises:

The video encoder encodes the luminance component to obtain the luminance code stream; obtains the encoded and reconstructed luminance component, the encoded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the chrominance block to be encoded; Determine the target chrominance prediction method with the smallest rate-distortion cost among the two chrominance prediction methods; wherein, the at least two chrominance prediction methods include a first-type chrominance prediction method and a second-type chrominance prediction method. The first type of chrominance prediction method is the intra-frame chrominance prediction method according to any one of claims 1 to 3; the target chrominance prediction method is generated through the association relationship between the chrominance prediction method and the indication information Indication information; subtracting the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is the color of the target chrominance prediction mode Chrominance information obtained after degree prediction; encoding the indication information and the chrominance residual error to obtain a chrominance code stream, and combining the chrominance code stream and the luminance code stream to obtain a video code stream;

The video decoder obtains the video code stream; decodes the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and the indication information; The target chrominance prediction mode is determined in the at least two chrominance prediction modes; according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, the target chrominance prediction mode The chrominance component performs chrominance prediction to obtain the chrominance prediction result; the chrominance reconstruction is performed according to the residual error obtained after decoding the chrominance residual information in the video bitstream and the chrominance prediction result to obtain the output chrominance .
A video encoding and decoding system, characterized in that it comprises a video encoder and a video decoder;

The video encoder is used to encode the luminance component to obtain the luminance code stream; obtain the encoded and reconstructed luminance component, the encoded and reconstructed adjacent chrominance information, and the original chrominance information corresponding to the chrominance block to be encoded; pass rate distortion Optimizing the determination of a target chrominance prediction method with the smallest rate-distortion cost from at least two chrominance prediction methods; wherein the at least two chrominance prediction methods include a first-type chrominance prediction method and a second-type chrominance prediction method The first type of chrominance prediction method is the intra-frame chrominance prediction method according to any one of claims 1 to 3; the target chrominance is generated through the association relationship between the chrominance prediction method and the indication information Indication information of the prediction mode; subtracting the original chrominance information and the predicted chrominance information to obtain chrominance residual information; wherein the predicted chrominance information is predicted by the target chrominance The chrominance information obtained after chrominance prediction is performed in a manner; the indication information and the chrominance residual are encoded to obtain a chrominance code stream, and the chrominance code stream and the luminance code stream are combined to obtain a video code stream ；

The video decoder is used to obtain the video code stream; decode the video code stream to obtain the decoded and reconstructed luminance component, the decoded and reconstructed adjacent chrominance information, and the indication information; according to the indication information , Determining the target chrominance prediction mode from the at least two chrominance prediction modes; according to the decoded and reconstructed luminance component and the decoded and reconstructed adjacent chrominance information, predict the target chrominance The chrominance component is predicted by the chrominance component to obtain the chrominance prediction result; the chrominance reconstruction is performed according to the residual error obtained after decoding the chrominance residual information in the video bitstream and the chrominance prediction result to obtain Output chromaticity.
A terminal device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claim 1 to 3. Steps of any one of the intra-frame chroma prediction methods.
A video encoder, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claim 4 Steps of the intra-frame chroma prediction method described in any one of ~6.
A video decoder, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program as claimed in claim 7 Steps of the intra-frame chroma prediction method described in any one of ~8.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the frame described in any one of claims 1 to 3 The steps of the internal chromaticity prediction method.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to realize the frame described in any one of claims 4 to 6 The steps of the internal chromaticity prediction method.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the frame according to any one of claims 7 to 8 is realized. The steps of the internal chromaticity prediction method.