CN115086679A

CN115086679A - Intra-frame prediction method and device, electronic equipment and storage medium

Info

Publication number: CN115086679A
Application number: CN202210642288.7A
Authority: CN
Inventors: 黄博; 闻兴; 于冰; 谷嘉文; 刘晶; 黄跃
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2022-06-08
Filing date: 2022-06-08
Publication date: 2022-09-20

Abstract

The present disclosure relates to an intra prediction method, an intra prediction apparatus, an electronic device, and a storage medium, wherein the method comprises: determining a target intra prediction mode and reference pixels for a target image block in a video frame; acquiring a reconstructed pixel value of a target image block in a target intra-frame prediction mode, and predicting through the reconstructed pixel value of the target image block to obtain first residual information of other image blocks; predicting through the original pixel value of the target image block to obtain second residual error information of other image blocks; determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, the original pixel value and the reconstructed pixel value of the reference pixel; and compensating the pixel value of the reference pixel according to the compensation mode, and performing intra-frame prediction on other image blocks influenced by the target image block based on the compensated reference pixel value. According to the method, the distortion of the reference pixel is reduced by compensating the pixel value of the reference pixel, so that the accuracy of the intra-frame prediction result is improved.

Description

Intra-frame prediction method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of video coding technologies, and in particular, to an intra prediction method, an intra prediction apparatus, an electronic device, a storage medium, and a program product.

Background

Video coding refers to a technology for converting one video format into another video format, and the purpose of video coding is to compress video data, reduce data size, and meet the requirements of storage and transmission. The intra-frame prediction is a main prediction method in the existing coding standard, and an image block which is not yet coded is predicted by using reconstructed pixels in an image block which is coded in a current frame by dividing each frame image of a video into square image blocks with fixed sizes as basic units.

However, since the video encoding process compresses and quantizes the video, discarding some unnecessary data that has less impact on the reconstruction quality, reconstructed pixels of the encoded image block may be distorted to different degrees. Since the subsequent uncoded image block uses the reconstructed pixel in the coded image block as the reference pixel to perform the prediction of the current uncoded image block, the distortion of the reference pixel will affect the prediction accuracy of the subsequent uncoded image block, thereby reducing the accuracy of the intra prediction result.

Disclosure of Invention

The present disclosure provides an intra prediction method, apparatus, electronic device, storage medium, and program product, to at least solve the problem in the related art that the distortion magnitude of a reference pixel will affect the prediction accuracy of a subsequent un-encoded image block, thereby reducing the accuracy of an intra prediction result. The technical scheme of the disclosure is as follows:

according to a first aspect of embodiments of the present disclosure, there is provided an intra prediction method, including:

determining a target intra prediction mode and reference pixels for a target image block in a video frame; the target image block is any one of a plurality of image blocks obtained after the video frame is divided, the reference pixels represent pixels influencing intra-frame prediction results of other image blocks in the target image block, and the other image blocks represent image blocks which are adjacent to the target image block and of which the pixel values are influenced by the target image block;

acquiring a reconstructed pixel value of the target image block in the target intra-frame prediction mode, and predicting through the reconstructed pixel value of the target image block to obtain first residual error information of other image blocks;

predicting through the original pixel value of the target image block to obtain second residual error information of the other image blocks;

determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel;

and compensating the pixel value of the reference pixel according to the compensation mode, and performing intra-frame prediction on other image blocks influenced by the target image block based on the compensated reference pixel value.

In an exemplary embodiment, the predicting first residual information of the other image blocks by the reconstructed pixel values of the target image block includes:

performing intra-frame prediction on the other image blocks through the reconstructed pixel values of the target image block to obtain first prediction pixel values of the other image blocks;

and obtaining first residual error information of the other image blocks based on the first predicted pixel value and the original pixel values of the other image blocks.

In an exemplary embodiment, the predicting second residual information of the other image blocks through original pixel values of the target image block includes:

performing intra-frame prediction on the other image blocks through the original pixel values of the target image block to obtain second predicted pixel values of the other image blocks;

and obtaining second residual error information of the other image blocks based on the second predicted pixel values and the original pixel values of the other image blocks.

In an exemplary embodiment, the determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel includes:

performing subtraction on the first residual information and the second residual information to obtain a first generation value for the other image blocks, and performing subtraction on an original pixel value of the reference pixel and a reconstructed pixel value of the reference pixel to obtain a second generation value for the reference pixel;

determining a compensation mode for a reference pixel in the target image block based on the first cost value for the other image block and the second cost value for the reference pixel.

In an exemplary embodiment, the reference pixels include a rightmost pixel column and a bottommost pixel row of the target image block, and the other image blocks include a right image block of the target image block and a lower image block of the target image block;

the determining a compensation mode for a reference pixel in the target image block based on the first cost value for the other image block and the second cost value for the reference pixel comprises:

acquiring a first generation value of the right image block, a first generation value of the lower image block, a second generation value of the rightmost pixel column and a second generation value of the bottommost pixel row;

determining a compensation mode for the reference pixel as a right side compensation mode when the first cost value of the right side image block is greater than a first threshold and the second cost value of the rightmost pixel column is greater than a second threshold;

determining a compensation mode for the reference pixel as a lower compensation mode if the first cost value of the lower image block is greater than a first threshold and the second cost value of the lowermost pixel row is greater than a second threshold;

determining a compensation mode for the reference pixel as a bilateral compensation mode when the first cost values of the right image block and the lower image block are both greater than a first threshold and the second cost values of the rightmost pixel column and the bottommost pixel row are both greater than a second threshold; the double-sided compensation mode includes the right-side compensation mode and the lower-side compensation mode.

In an exemplary embodiment, the subtracting the first residual information and the second residual information to obtain a first cost value for the other image blocks includes:

performing subtraction on the first residual error information and the second residual error information to obtain a first initial cost value for the other image blocks;

acquiring a weight value aiming at the reference pixel; the weight represents the influence degree of the reference pixel on the intra-frame prediction results of the other image blocks;

and performing weighting processing on the first initial cost value through the weight value to obtain a first generation value for the other image blocks.

In an exemplary embodiment, the obtaining the weight value for the reference pixel includes:

performing texture detection on the other image blocks to obtain texture directions of the other image blocks;

and determining the weight value aiming at the reference pixel based on the angle corresponding to the texture direction.

In an exemplary embodiment, the determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel further includes:

predicting to obtain third residual error information of the target image block through an original pixel value of the video frame;

determining a compensation mode for the reference pixel as uncompensated if it is determined that the first, second, third, original, and reconstructed pixel values of the reference pixel do not satisfy a preset compensation condition.

In an exemplary embodiment, the compensating the pixel value of the reference pixel according to the compensation mode includes:

determining a compensation value calculation relation for the reference pixel according to the compensation mode, and determining a compensation pixel value for the reference pixel based on the compensation value calculation relation;

compensating a pixel value of the reference pixel based on the compensated pixel value.

In an exemplary embodiment, the determining the target intra prediction mode for the target image block in the video frame includes:

determining the rate distortion cost value of the target image block under various candidate intra-frame prediction modes through a rate distortion cost estimation model; the rate-distortion estimation model comprises a first cost estimation unit for the other image block and a second cost estimation unit for a reference pixel in the target image block;

and determining the candidate intra-frame prediction mode corresponding to the rate distortion cost value with the minimum value in the rate distortion cost values as the target intra-frame prediction mode.

According to a second aspect of the embodiments of the present disclosure, there is provided an intra prediction apparatus, including:

a prediction mode determination unit configured to perform determining a target intra prediction mode and a reference pixel for a target image block in a video frame; the target image block is any one of a plurality of image blocks obtained after the video frame is divided, the reference pixel represents a pixel which affects intra-frame prediction results of other image blocks in the target image block, and the other image blocks represent image blocks which are adjacent to the target image block and have pixel values affected by the target image block;

a first residual obtaining unit configured to perform obtaining of a reconstructed pixel value of the target image block in the target intra prediction mode, and obtain first residual information of the other image blocks through prediction of the reconstructed pixel value of the target image block;

a second residual obtaining unit configured to perform second residual information of the other image blocks obtained by predicting original pixel values of the target image block;

a compensation mode determination unit configured to perform determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel;

and the intra-frame prediction unit is configured to perform compensation on the pixel value of the reference pixel according to the compensation mode, and perform intra-frame prediction on other image blocks influenced by the target image block based on the compensated reference pixel value.

In an exemplary embodiment, the first residual obtaining unit is specifically configured to perform intra prediction on the other image blocks through reconstructed pixel values of the target image block to obtain first predicted pixel values of the other image blocks; and obtaining first residual error information of the other image blocks based on the first predicted pixel value and the original pixel values of the other image blocks.

In an exemplary embodiment, the second residual obtaining unit is specifically configured to perform intra prediction on the other image blocks through original pixel values of the target image block to obtain second predicted pixel values of the other image blocks; and obtaining second residual error information of the other image blocks based on the second predicted pixel values and the original pixel values of the other image blocks.

In an exemplary embodiment, the compensation mode determining unit is specifically configured to perform a subtraction of the first residual information and the second residual information to obtain a first cost value for the other image blocks, and a subtraction of an original pixel value of the reference pixel and a reconstructed pixel value of the reference pixel to obtain a second cost value for the reference pixel; determining a compensation mode for a reference pixel in the target image block based on the first cost value for the other image block and the second cost value for the reference pixel.

In an exemplary embodiment, the reference pixels include a rightmost pixel column and a bottommost pixel row of the target image block, and the other image blocks include a right image block of the target image block and a lower image block of the target image block; the compensation mode determination unit is further configured to perform acquiring a first cost value of the right image block, a first cost value of the lower image block, a second cost value of the rightmost pixel column, and a second cost value of the bottommost pixel row; determining a compensation mode for the reference pixel as a right side compensation mode when the first cost value of the right side image block is greater than a first threshold and the second cost value of the rightmost pixel column is greater than a second threshold; determining a compensation mode for the reference pixel as a lower compensation mode if the first cost value of the lower image block is greater than a first threshold and the second cost value of the lowermost pixel row is greater than a second threshold; determining a compensation mode for the reference pixel as a bilateral compensation mode when the first cost values of the right image block and the lower image block are both greater than a first threshold and the second cost values of the rightmost pixel column and the bottommost pixel row are both greater than a second threshold; the double-sided compensation mode includes the right-side compensation mode and the lower-side compensation mode.

In an exemplary embodiment, the compensation mode determining unit is further configured to perform subtraction on the first residual information and the second residual information to obtain a first initial cost value for the other image blocks; acquiring a weight value aiming at the reference pixel; the weight represents the influence degree of the reference pixel on the intra-frame prediction results of the other image blocks; and performing weighting processing on the first initial cost value through the weight value to obtain a first generation value for the other image blocks.

In an exemplary embodiment, the compensation mode determining unit includes a weight determining subunit configured to perform texture detection on the other image blocks to obtain texture directions of the other image blocks; and determining the weight value aiming at the reference pixel based on the angle corresponding to the texture direction.

In an exemplary embodiment, the compensation mode determining unit is further configured to perform a third residual information of the target image block predicted from an original pixel value of the video frame; determining a compensation mode for the reference pixel as uncompensated if it is determined that the first, second, third, original, and reconstructed pixel values of the reference pixel do not satisfy a preset compensation condition.

In an exemplary embodiment, the intra prediction unit is specifically configured to perform determining a compensation value calculation relation for the reference pixel according to the compensation mode, and determining a compensation pixel value for the reference pixel based on the compensation value calculation relation; compensating a pixel value of the reference pixel based on the compensated pixel value.

In an exemplary embodiment, the prediction mode determination unit is specifically configured to perform determining, by a rate-distortion cost estimation model, rate-distortion cost values of the target image block in a plurality of candidate intra prediction modes; the rate-distortion estimation model comprises a first cost estimation unit for the other image block and a second cost estimation unit for a reference pixel in the target image block; and determining the candidate intra-frame prediction mode corresponding to the rate distortion cost value with the minimum value in the rate distortion cost values as the target intra-frame prediction mode.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the method of any one of the above.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the method of any one of the above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising instructions which, when executed by a processor of an electronic device, enable the electronic device to perform the method as defined in any one of the above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

the method includes the steps of conducting intra-frame prediction on other image blocks through an original pixel value and a reconstructed pixel value of a target image block respectively to obtain first residual error information and second residual error information, determining a compensation mode aiming at a reference pixel in the target image block according to the first residual error information, the second residual error information, the original pixel value and the reconstructed pixel value of the reference pixel, and then compensating the pixel value of the reference pixel according to the reference mode to enable the pixel value of the reference pixel to be closer to the original pixel value, reducing distortion of the reference pixel and improving accuracy of an intra-frame prediction result.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1(a) is a schematic illustration of image content shown according to an exemplary embodiment.

Fig. 1(b) is a diagram illustrating intra prediction according to an example embodiment.

Fig. 2 is a flowchart illustrating a method of intra prediction according to an example embodiment.

Fig. 3 is a diagram illustrating 34 candidate intra prediction modes according to an example embodiment.

Fig. 4 is a flowchart illustrating a method of intra prediction according to another exemplary embodiment.

Fig. 5 is an overall flowchart illustrating a method of compensating intra prediction and encoding based on mode dependency and reference pixels according to an exemplary embodiment.

Fig. 6 is a schematic diagram illustrating a compensation sequence according to a compensated pixel value according to an exemplary embodiment.

Fig. 7 is a block diagram illustrating a structure of an intra prediction apparatus according to an exemplary embodiment.

FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for presentation, analyzed data, etc.) referred to in the present disclosure are both information and data that are authorized by the user or sufficiently authorized by various parties.

Existing Coding standards, such as the international standard HEVC (High Efficiency Video Coding), VVC (universal Video Coding), and the like, include two prediction methods, intra and inter. For example, the HEVC standard still adopts a block-based hybrid coding framework, and first, each frame image of a video needs to be divided into square image blocks of fixed size as a basic unit, and each image block is sequentially coded according to a raster order. The image block is divided into coding blocks, each coding block uses a reference image to perform intra/inter prediction, and the difference between a prediction block and an original block is a residual block. And the generated residual block is sequentially transformed and quantized, and is entropy-coded together with a coding mode to form a code stream. The predicted residual error is usually much smaller in amplitude than the original pixel value, so that the coding efficiency can be greatly improved by using the coding pixel difference value to replace the direct coding of the original pixel value. In order to provide prediction reference for the subsequent coding block, the quantized residual block is further processed by inverse transformation, inverse quantization and loop filtering to finally generate a reconstructed image, the generated reconstructed image provides reference for the intra-frame or inter-frame prediction process, and the process of generating the reconstructed image is also part of the decoding process.

Referring to fig. 1(a) for a schematic diagram of intra prediction in an exemplary embodiment, the intra prediction is to use reconstructed pixels already encoded in a current frame to predict an image block that is not yet encoded, so as to remove spatial redundancy in a video, and positions of reference pixels used in different intra prediction modes are different during intra prediction, for example, HEVC standard is taken as an example, in fig. 1(b), a horizontal mode (mode 10) uses a column of reconstructed pixels on the left side of the current block as reference pixels (left diagonally shaded pixels in the lower drawing), and a vertical mode (mode 26) uses a row of reconstructed pixels above the current block as reference pixels (diagonally shaded pixels above the lower drawing).

As can be known from the above coding method, in lossy video compression, the reference pixels are usually subjected to quantization process, so there will be distortion in different degrees. Since the subsequent blocks use the encoded reconstructed pixels as reference pixels for the prediction of the current block, it is obvious that the distortion magnitude of the reference pixels affects the prediction accuracy of the current block.

In view of the above problems, the present disclosure provides an intra prediction and encoding method based on mode dependency and reference pixel compensation, which uses RDO (Rate-distortion optimization) to judge and compensate distortion of a reference pixel, so as to reduce distortion of the reference pixel as much as possible and improve video compression efficiency.

Referring to fig. 2, a flowchart of an intra prediction method according to an exemplary embodiment is shown, where this embodiment is illustrated as applying the method to a terminal, it is understood that this method may also be applied to a server, and may also be applied to a system including the terminal and the server, and is implemented through interaction between the terminal and the server. The terminal can be but not limited to various personal computers, notebook computers, smart phones, tablet computers, internet of things equipment and portable wearable equipment, and the internet of things equipment can be smart sound boxes, smart televisions, smart air conditioners, smart vehicle-mounted equipment and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The server may be implemented as a stand-alone server or as a server cluster consisting of a plurality of servers. In this embodiment, the method includes the steps of:

in step S210, a target intra prediction mode and reference pixels for a target image block in a video frame are determined; the target image block is any one of a plurality of image blocks obtained after a video frame is divided, the reference pixel represents a pixel which affects intra-frame prediction results of other image blocks in the target image block, and the other image blocks represent image blocks which are adjacent to the target image block and have pixel values affected by the target image block.

The target intra-frame prediction mode represents an optimal intra-frame prediction mode when the target image block is used as a reference image block, and specifically may be an intra-frame prediction mode with a minimum rate distortion cost value in multiple intra-frame prediction modes corresponding to the encoder.

The reference pixels of the target image block may be understood as pixels in the target image block used for intra prediction of other image blocks when the target image block is used as a reference image block.

In a specific implementation, referring to fig. 1(a), when performing intra-frame prediction, prediction is performed based on the lowest pixel row or the rightmost pixel column of the reference image block, and thus, the reference pixels of the target image block may include the rightmost pixel column in the target image block and the lowest pixel row in the target image block. When determining the target intra prediction mode of the target image block, the candidate intra prediction modes are determined based on the type of the encoder, for example, if the HEVC encoder is used for encoding, 35 corresponding candidate intra prediction modes include a dc mode (a non-directional prediction mode in which the pixel values of all prediction pixels are equal to the average value of the pixel value of the reference pixel), a planer mode (a non-directional prediction mode in which the pixel values of the prediction pixels are obtained by interpolation of the reference pixel), and 33 angular prediction modes, and referring to fig. 3, fig. 3 shows specific directions of the 33 angular prediction modes, where h and v in fig. 3 respectively represent the horizontal direction and the vertical direction, and prediction directions of the remaining angular prediction modes except h and v, i.e., v-8, v-7 … v +8, h-7 in fig. 3, h-6 … h +8 can all be considered as an offset in the vertical or horizontal direction. And then calculating rate distortion cost values of the target image block in each candidate intra-frame prediction mode through a preset rate distortion cost estimation model, determining the rate distortion cost value with the minimum value from each rate distortion cost value, and determining the candidate intra-frame prediction mode corresponding to the rate distortion cost value with the minimum value as the target intra-frame prediction mode. Wherein the rate-distortion estimation model comprises a first cost estimation unit for the other image blocks and a second cost estimation unit for the reference pixels in the target image block, and therefore the determined rate-distortion cost values in the various modes also comprise the first cost values for the other image blocks and the second cost values for the reference pixels in the target image block.

In step S220, a reconstructed pixel value of the target image block in the target intra prediction mode is obtained, and first residual information of other image blocks is obtained through prediction of the reconstructed pixel value of the target image block.

The first residual information may be an SATD (Sum of Absolute Transformed Difference) value of a first residual, and the first residual information represents residual information between the first predicted pixel value of the other image block and the original pixel value of the other image block.

Wherein the reconstructed pixel value represents a pixel value of a reconstructed image generated based on the target image block. The reconstructed image is generated by performing inverse transformation, inverse quantization and loop filtering on a quantized residual block corresponding to a target image block in a target intra-frame prediction mode, the quantized residual block is obtained by performing transformation and quantization on the residual block of the target image block, and the residual block of the target image block represents a difference value between a predicted image block and an original image block of the target image block after intra-frame prediction is performed on the target image block in the target intra-frame prediction mode to obtain the predicted image block.

In the specific implementation, after a target image block is determined, intra-frame prediction is performed on the target image block in a target intra-frame prediction mode to obtain a predicted image block, a residual block is obtained based on the predicted image block and an original image block of the target image block, the residual block is subjected to change and quantization processing to obtain a quantized residual block, inverse transformation, inverse quantization and loop filtering are performed on the quantized residual block to generate a reconstructed image corresponding to the target image block, and pixel values of pixel points in the reconstructed image are reconstructed pixel values representing the reconstructed pixel values of the target image block in the target intra-frame prediction mode. And further performing intra-frame prediction on other image blocks through the reconstructed pixel value of the target image block to obtain a first predicted pixel value, obtaining a first residual of the other image blocks based on the predicted first predicted pixel value and the original pixel values of the other image blocks, and calculating the SATD value of the first residual to be used as first residual information of the other image blocks.

In step S230, second residual information of other image blocks is obtained by predicting the original pixel value of the target image block.

The second residual information may be an SATD value of the second residual, and the second residual information represents residual information between second predicted pixel values of other image blocks and original pixel values of other image blocks.

In the specific implementation, intra-frame prediction is performed on other image blocks through original pixel values of a target image block to obtain second predicted pixel values, second residuals of the other image blocks are obtained based on the second predicted pixel values obtained through prediction and the original pixel values of the other image blocks, and SATD values of the second residuals are calculated to serve as second residual information of the other image blocks.

In step S240, a compensation mode for the reference pixel in the target image block is determined based on the first residual information, the second residual information, the original pixel value of the reference pixel, and the reconstructed pixel value of the reference pixel.

The compensation modes may include a right compensation mode, a lower compensation mode, a double-sided compensation mode and no compensation, where the right compensation mode represents compensation for a rightmost pixel column in the target image block, the lower compensation mode represents compensation for a bottommost pixel row in the target image block, and the double-sided compensation mode represents compensation for both the rightmost pixel column and the bottommost pixel row in the target image block.

In specific implementation, based on the first residual error information and the second residual error information, a first generation value for other image blocks can be obtained; based on the original pixel value of the reference pixel and the reconstructed pixel value of the reference pixel, a second cost value for the reference pixel in the target image block may be obtained. And comparing the first generation value and the second generation value with preset compensation conditions, and determining a compensation mode aiming at the reference pixel in the target image block according to the comparison result.

More specifically, when intra prediction is performed, prediction is performed based on the lowermost pixel row or the rightmost pixel column of the reference image block, and therefore, the position of the reference pixel in the target image block is the lowermost pixel row or the rightmost pixel column, and correspondingly, the other image blocks may include the right image block and the lower image block, and therefore, the right image block and the lower image block both have respective corresponding first generation values, and the lowermost pixel row and the rightmost pixel column both have respective corresponding second generation values.

In step S250, the pixel value of the reference pixel is compensated according to the compensation mode, and intra prediction is performed on other image blocks affected by the target image block based on the compensated reference pixel value.

In specific implementation, different compensation modes correspond to different compensation value calculation relational expressions, after a compensation mode for a reference pixel in a target image block is determined, a compensation pixel value of the reference pixel is determined according to the calculation relational expression corresponding to the compensation mode, and then the pixel value of the target reference pixel is compensated according to the compensation pixel value, so that the pixel value of the reference pixel is close to an original pixel of the target image block, the compensated reference pixel value is obtained, and when other image blocks influenced by the target image block are subjected to intra-frame prediction, the intra-frame prediction can be performed based on the compensated reference pixel value.

In the intra-frame prediction method, intra-frame prediction is performed on other image blocks through an original pixel value and a reconstructed pixel value of a target image block to obtain first residual error information and second residual error information, a compensation mode for a reference pixel in the target image block is determined according to the first residual error information, the second residual error information, the original pixel value and the reconstructed pixel value of the reference pixel, and then the pixel value of the reference pixel is compensated according to the compensation mode, so that the pixel value of the reference pixel is closer to the original pixel value, the distortion of the reference pixel is reduced, and the accuracy of an intra-frame prediction result is improved.

In an exemplary embodiment, in step S220, the first residual information of the other image blocks is obtained through prediction of the reconstructed pixel value of the target image block, which may specifically be implemented through the following steps:

step 2201, intra-frame prediction is carried out on other image blocks through the reconstructed pixel values of the target image block to obtain first prediction pixel values of the other image blocks;

step S2202 obtains first residual information of the other image block based on the first predicted pixel value and the original pixel values of the other image block.

In the specific implementation, intra-frame prediction is performed on other image blocks through reconstructed pixel values of a target image block to obtain a first predicted pixel value, the first predicted pixel value obtained through prediction and original pixel values of the other image blocks are subjected to subtraction to obtain first residuals of the other image blocks, and SATD values of the first residuals are further calculated to serve as first residual information of the other image blocks.

More specifically, because the image block includes a plurality of pixel points, when the difference is made between the first predicted pixel value and the original pixel value of the other image block, the difference is made between the first predicted pixel value and the original pixel value of each pixel point in the other image block to obtain a first residual error of each pixel point, the first residual error of each pixel point forms a residual error matrix, which is recorded as a first residual error matrix, and the size of the first residual error matrix is the same as that of the other image block. Calculating the SATD value of the first residual, namely calculating the SATD value of the first residual matrix, specifically, obtaining the changed residual of each pixel after the pixel is subjected to Hadamard transform, and summing the absolute values of the changed residuals of the pixels to obtain the SATD value of the first residual.

In this embodiment, intra prediction is performed on other image blocks through the reconstructed pixel value of the target image block to obtain first residual information of the other image blocks, so that the first residual information and the second residual information are combined to determine a compensation mode for the reference pixel.

In an exemplary embodiment, in step S230, the second residual information of other image blocks is obtained by predicting the original pixel value of the target image block, which may specifically be implemented by the following steps:

step S2301, intra-frame prediction is carried out on other image blocks through original pixel values of the target image block, and second prediction pixel values of the other image blocks are obtained;

step S2302, obtaining second residual information of the other image blocks based on the second predicted pixel values and the original pixel values of the other image blocks.

In the specific implementation, intra-frame prediction is performed on other image blocks through original pixel values of a target image block to obtain second predicted pixel values, the second predicted pixel values obtained through prediction and the original pixel values of the other image blocks are subjected to subtraction to obtain second residuals of the other image blocks, and SATD values of the second residuals are further calculated to serve as second residual information of the other image blocks.

More specifically, because the image block includes a plurality of pixel points, when the second predicted pixel value and the original pixel value of the other image block are subtracted, the second predicted pixel value and the original pixel value of each pixel point in the other image block are subtracted to obtain a second residual error of each pixel point, the second residual error of each pixel point forms a residual error matrix, which is recorded as a second residual error matrix, and the size of the second residual error matrix is the same as that of the other image block. And calculating the SATD value of the second residual error, namely calculating the SATD value of the second residual error matrix, specifically, obtaining the changed residual error of each pixel point after the pixel points are subjected to Hadamard transform, and summing the absolute values of the changed residual errors of the pixel points to obtain the SATD value of the second residual error.

In this embodiment, intra prediction is performed on other image blocks through the original pixel values of the target image block to obtain second residual information of the other image blocks, so that the compensation mode for the reference pixel is determined by combining the second residual information with the first residual information.

In an exemplary embodiment, in step S240, a compensation mode for a reference pixel in the target image block is determined based on the first residual information, the second residual information, the original pixel value of the reference pixel, and the reconstructed pixel value of the reference pixel, which may specifically be implemented by the following steps:

in step S2401, performing subtraction on the first residual information and the second residual information to obtain a first generation value for the other image block, and performing subtraction on an original pixel value of the reference pixel and a reconstructed pixel value of the reference pixel to obtain a second generation value for the reference pixel;

in step S2402, a compensation mode for the reference pixel in the target image block is determined based on the first cost value for the other image blocks and the second cost value for the reference pixel.

The reference pixel includes a plurality of pixel points, and therefore, the second generation value may be an SAD value (Sum of Absolute differences) of the reference pixel).

In a specific implementation, the other image blocks may include a right image block and a lower image block, and therefore, both the right image block and the lower image block may be calculated to obtain the first residual information and the second residual information. And the reference pixels in the target image block may include the rightmost pixel column and the bottommost pixel row of the target image block, and thus, the rightmost pixel column and the bottommost pixel row of the target image block also have the original pixel value and the reconstructed pixel value, respectively.

Further, the first residual information and the second residual information of the right image block may be subtracted to obtain a first generation value for the right image block, and the first residual information and the second residual information of the lower image block may be subtracted to obtain a first generation value for the lower image block. And subtracting the original pixel value and the reconstructed pixel value of the lowest pixel row of the target image block to obtain a second generation value of the lowest pixel row. A compensation pattern for a reference pixel in the target image block is determined based on the first cost value for the right image block, the first cost value for the lower image block, the second cost value for the rightmost pixel column, and the second cost value for the bottommost pixel row.

For example, if the first residual information of the right image block R is SATD-R _{Reconstruction} The second residual information of the right image block is SATD-R _Original Then the first generation value of the right image block can be expressed as: [ SATD-R ] _{Reconstruction} －SATD-R _Original |。

If the first residual information of the lower image block B is recordedIs SATD-B _{Reconstruction} The second residual information of the lower image block is SATD-B _Original Then the first cost value of the lower image block can be expressed as: [ SATD-B ] _{Reconstruction} －SATD-B _Original |。

If the original pixel value of the rightmost pixel row is Pix-right _Original The reconstructed pixel value of the rightmost pixel column in the target intra prediction mode is Pix-right _{Reconstruction} Then the second cost value of the rightmost pixel column can be expressed as: SAD (Pix-right) _Original －Pix-right _{Reconstruction} )。

If the original pixel value of the bottom pixel row is Pix-bottom _Original The reconstructed pixel value of the lowest pixel row in the target intra prediction mode is Pix-bottom _{Reconstruction} Then the second cost value of the bottom-most pixel row can be expressed as: SAD (Pix-bottom) _Original －Pix-bottow _{Reconstruction} )。

It should be noted that, the first generation value and the second generation value in the present disclosure are used to represent the degree of deviation, and therefore, both the first generation value and the second generation value are positive numbers, that is, if the difference between the first residual information and the second residual information of the image block is less than 0, the absolute value of the difference needs to be obtained and then the obtained difference is used as the first generation value.

In this embodiment, a first generation value for the other image blocks is obtained by subtracting the first residual information and the second residual information of the other image blocks, a second generation value for the reference pixel is obtained by subtracting the original pixel value and the reconstructed pixel value of the reference pixel, and finally, a compensation mode for the reference pixel in the target image block is determined based on the first generation value and the second generation value, and the compensation mode is determined jointly by considering the reference pixel and two dimensions of the other image blocks affected by the reference pixel, so that accuracy of the determined compensation mode can be ensured.

In an exemplary embodiment, the reference pixels include a rightmost pixel column and a bottommost pixel row of the target image block, and the other image blocks include a right image block of the target image block and a lower image block of the target image block; thus, the first generation value of the right image block, the first generation value of the lower image block, the second generation value of the rightmost pixel column, and the second generation value of the bottommost pixel row can be obtained; in step S2402, determining a compensation mode for the reference pixel in the target image block based on the first generation value for the other image blocks and the second generation value for the reference pixel, which includes the following specific cases:

step S2402A, determining that the compensation mode for the reference pixel is the right-side compensation mode when the first cost value of the right-side image block is greater than the first threshold and the second cost value of the rightmost pixel column is greater than the second threshold;

step S2402B, determining that the compensation mode for the reference pixel is a lower side compensation mode when the first cost value of the lower image block is greater than the first threshold and the second cost value of the lowermost pixel row is greater than the second threshold;

step S2402C, determining that the compensation mode for the reference pixel is the bilateral compensation mode when the first cost values of the right image block and the lower image block are both greater than the first threshold and the second cost values of the rightmost pixel column and the bottommost pixel row are both greater than the second threshold; the double-sided compensation mode includes a right-side compensation mode and a lower-side compensation mode.

The first threshold and the second threshold may be equal or unequal.

In specific implementation, the first threshold is ThD, the second threshold is ThE, and the first cost value of the right image block is: [ SATD-R ] _{Reconstruction} －SATD-R _Original The first generation value of the lower image block is: [ SATD-B ] _{Reconstruction} －SATD-B _Original The second generation value of the rightmost pixel column is: SAD (Pix-right) _Original －Pix-right _{Reconstruction} ) The second cost value of the bottom pixel row is: SAD (Pix-bottom) _Original －Pix-bottow _{Reconstruction} ) Then, there are:

(1) the conditions when the compensation mode for the reference pixel is the right side compensation mode are: [ SATD-B ] _{Reconstruction} －SATD-B _Original |>ThD, and SAD (Pix-right) _Original －Pix-right _{Reconstruction} )>ThE。

(2) The conditions when the compensation mode for the reference pixel is the lower side compensation mode are: [ SATD-B ] _{Reconstruction} －SATD-B _Original |>ThD，SAD(Pix-bottow _Original －Pix-bottow _{Reconstruction} )>ThE。

(3) The condition when the compensation mode for the reference pixel is the double-sided compensation mode is: the conditions of (1) and (2) are satisfied simultaneously.

In this embodiment, the first generation values of the right image block and the lower image block, and the second generation values of the rightmost pixel column and the bottommost pixel row are matched with the preset compensation condition, so as to determine the compensation mode for the reference pixel, and the compensation mode is divided, so that the corresponding compensation pixel value is rapidly determined according to different compensation modes, thereby avoiding compensation errors, resulting in time waste and reducing the coding efficiency of the video frame.

In an exemplary embodiment, in the step S2401, the difference is performed on the first residual information and the second residual information to obtain the first cost value for the other image blocks, which may specifically be implemented by the following steps:

step S2401A, subtracting the first residual error information and the second residual error information to obtain a first initial cost value for the other image blocks;

step S2401B, obtaining a weight value for the reference pixel; the weight represents the influence degree of the reference pixel on the intra-frame prediction results of other image blocks;

step S2401C, performing weighting processing on the first initial cost value through the weight value to obtain a first generation value for the other image blocks.

In specific implementation, the determination of the cost values for other image blocks is affected by different degrees of influence of the reference pixel on the intra-frame prediction results of other image blocks, so that before the first generation value for other image blocks is determined, a weight value for the reference pixel needs to be obtained, and then the first residual information of other image blocks and the second residual information of other image blocks are subjected to subtraction to obtain a first initial cost value for other image blocks, and then the first initial cost value can be weighted by the weight value for the reference pixel to obtain the first generation value for other image blocks.

More specifically, when the reference pixel comprises the rightmost pixel column and the bottommost pixel row of the target image block, and other image blocks comprise the right image block of the target image block and the lower image block of the target image block, the weight of the rightmost pixel column and the weight of the bottommost pixel row are respectively obtained, the first initial cost value of the right image block is weighted through the weight of the rightmost pixel column to obtain a first generation value for the right image block, and the first initial cost value of the lower image block is weighted through the weight of the bottommost pixel row to obtain the first generation value for the lower image block.

For example, if the weight of the rightmost pixel row is K-R and the weight of the bottommost pixel row is K-B, the first generation value of the right image block is: K-R (| SATD-R) _{Reconstruction} －SATD-R _Original |), the first generation value of the following image block is: K-B (| SATD-B) _{Reconstruction} －SATD-B _Original |)。

In this embodiment, in consideration of the influence degree of the reference pixel on the intra-frame prediction results of other image blocks, a scheme of determining the weight of the reference pixel and correcting the first initial cost value of the other image blocks through the weight of the reference pixel is provided, so that the accuracy of the determined first cost value for the other image blocks can be improved, the accuracy of determining the compensation mode for the reference pixel according to the first cost value of the other image blocks is improved, and further, the distortion of the reference pixel can be compensated better.

In an exemplary embodiment, in step S2401B, obtaining the weight value for the reference pixel includes: performing texture detection on other image blocks to obtain texture directions of the other image blocks; and determining the weight value aiming at the reference pixel based on the angle corresponding to the texture direction.

In specific implementation, the texture directions of other image blocks can be determined by using a selection method of intra-frame prediction modes in coding standards HEVC and VVC, or the characteristics of textures can be analyzed by using an image gradient detection method (gradient angles are calculated after filtering by a sobel operator), and then the textures of each block are classified to determine the texture directions of other image blocks. For example: detecting edges by using a sobel operator, and when the average edge strength of other image blocks is greater than th, and obtaining the average value (0-180 degrees) of the overall gradient directions of other image blocks, classifying textures into 34 classes, as shown in fig. 3, where the classes in fig. 3 include 33 texture directions and one class without obvious textures (i.e., the average edge strength is less than th, such as a pure white and pure black image block). After the texture directions of other image blocks are determined, the texture directions can be input into a preset weight estimation model to obtain a weight for a reference pixel, wherein the weight estimation model can be expressed as: where "K" is clip3(0, X, abs (cot (ang-M)), where "ang-M" denotes an angle value corresponding to a texture direction of another image block, "cot" denotes a cotangent function, "abs" denotes a function for obtaining an absolute value, and X is a preset number less than 5, for example, X "2". the clip3 function is to limit the value of "abs (cot (M)" to a range of 0-X, that is, to make it equal to "X" when the value of "abs (cot (ang-M)" is greater than "X", and to make it equal to "0" when the value of "abs (cot (ang-M)" is less than "0".

More specifically, when the other image block is the right image block, the corresponding reference pixel is the rightmost pixel row, and the weight estimation model may be: K-R is clip3(0, X, abs (cot (ang-M-R)).

When the other image blocks are the lower image blocks, the weight estimation model of the corresponding lowest pixel row may be: K-B is clip3(0, X, abs (cot (ang-M-B)).

Therefore, other image blocks can obtain the reference pixel weight: { K-R, K-B }.

In this embodiment, the weight for the reference pixel is determined according to the texture direction obtained by performing texture detection on the other image blocks, so that the first initial cost values of the other image blocks are corrected according to the weight of the reference pixel, and the accuracy of the determined first generation value for the other image blocks is improved.

In an exemplary embodiment, in step S240, determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, the original pixel value of the reference pixel, and the reconstructed pixel value of the reference pixel, further includes: predicting through an original pixel value of the video frame to obtain third residual error information of the target image block; and under the condition that the first residual error information, the second residual error information, the third residual error information, the original pixel value of the reference pixel and the reconstructed pixel value of the reference pixel are determined not to meet the preset compensation condition, determining that the compensation mode aiming at the reference pixel is not compensated.

The third residual information may be an SATD value of a third residual, and the third residual information represents residual information between a third predicted pixel value of the target image block and an original pixel value of the target image block.

In a specific implementation, when determining the compensation mode for the reference pixel, the compensation mode for the reference pixel may be determined jointly in combination with the third residual information of the target image block on the basis of the first residual information, the second residual information, the original pixel value of the reference pixel, and the reconstructed pixel value of the reference pixel. The method for obtaining third residual error information of the target image block through predicting the original pixel value of the video frame comprises the following steps: the method comprises the steps of firstly determining a reference image block influencing a target image block, conducting intra-frame prediction on the target image block through an original pixel value of the reference image block serving as the target image block in a video frame to obtain a predicted pixel value of the target image block, subtracting the predicted pixel value of the target image block from the original pixel value of the target image block to obtain a third residual error of the target image block, and calculating an SATD value of the third residual error to obtain third residual error information of the target image block. The specific determination method of the third residual information is similar to that of the first residual information and the second residual information, and is not repeated here.

More specifically, the condition that the compensation mode of the reference pixel is not compensated may be: the third residual information is greater than a third threshold, or the second cost value of the rightmost pixel column is less than the second threshold, or the second cost value of the bottommost pixel row is less than the second threshold, or the sum of the first cost value of the right image block and the second cost value of the lower image block is less than a fourth threshold.

For example, if the third residual information of the target image block i is SATD-i, the third threshold is ThA, and the fourth threshold is ThB, the compensation is performedThe condition that the mode is not compensated is that any one of the following conditions is satisfied: (1) SATD-i>ThA；(2)SAD(Pix-right _Original －Pix-right _{Reconstruction} )<ThE；(3)SAD(Pix-bottow _{Original (original)} －Pix-bottow _{Reconstruction} )<ThE；(4)K-R*(|SATD-R _{Reconstruction} －SATD-R _Original |)+K-B*(|SATD-B _{Reconstruction} －SATD-B _Original |)<ThB。

In this embodiment, the third residual information of the target image block is introduced, and the compensation mode for the reference pixel is determined according to the first residual information, the second residual information, the original pixel value of the reference pixel, and the reconstructed pixel value of the reference pixel.

In an exemplary embodiment, in step S250, compensating the pixel value of the reference pixel according to the compensation mode includes:

step S2501, determining a compensation value calculation relational expression aiming at the reference pixel according to the compensation mode, and determining a compensation pixel value aiming at the reference pixel based on the compensation value calculation relational expression;

in step S2502, the pixel value of the reference pixel is compensated based on the compensation pixel value.

In a specific implementation, a calculation relation of the compensation value corresponding to the right-side compensation mode is as follows: original pixel value of rightmost pixel column — reconstructed pixel value of rightmost pixel column; the calculation relation of the compensation value corresponding to the lower compensation mode is as follows: original pixel value of the lowermost pixel row-reconstructed pixel value of the lowermost pixel row; the calculation relation of the compensation value corresponding to the double-side compensation mode is as follows: original pixel values of the rightmost pixel column-reconstructed pixel values of the rightmost pixel column, and original pixel values of the lowermost pixel row-reconstructed pixel values of the lowermost pixel row.

After the compensation mode for the reference pixel is determined, the compensation pixel value for the reference pixel is calculated according to the corresponding compensation value calculation relation, and then the reference pixel can be compensated according to the compensation pixel value, so that the pixel value of the reference pixel is closer to the original pixel value of the reference pixel, and the distortion of the reference pixel is reduced.

In this embodiment, a compensation value calculation relation for the reference pixel is determined through the compensation mode, a compensation pixel value for the reference pixel is determined based on the compensation value calculation relation, and the reference pixel is compensated based on the compensation pixel value, so that the pixel value of the reference pixel is closer to the original pixel value of the reference pixel, distortion of the reference pixel is reduced, accuracy of intra-frame prediction results of other image blocks predicted based on the reference pixel is improved, and quality of video coding is improved.

In an exemplary embodiment, in step S210, determining a target intra prediction mode for a target image block in a video frame includes:

step S2101, determining the rate distortion cost value of the target image block in various candidate intra-frame prediction modes through a rate distortion cost estimation model; the rate-distortion estimation model comprises a first cost estimation unit for other image blocks and a second cost estimation unit for reference pixels in the target image block;

step S2102 determines the candidate intra prediction mode corresponding to the rate distortion cost value with the smallest value among the rate distortion cost values as the target intra prediction mode.

In a specific implementation, if a target image block is i, a right image block of the target image block is R, and a lower image block is B, a rate-distortion cost estimation model under any candidate intra prediction mode for the target image block i can be expressed as: j (i-mode-J) ═ SSE (mode-J) + lambda R (mode-J) + w ₁ *RD-RB(mode-j)+w ₂ *RD-rec(mode-j)

Wherein J (i-mode-J) represents the rate-distortion cost of the target image block i at mode-J (J may be any one of intra prediction mode indices 0-34); SSE represents the sum of squares of the difference values of the reconstructed pixel and the original pixel of the target image block i in the candidate intra-frame prediction mode-j; r (mode-j) represents the code rate of the candidate intra-frame prediction mode-j, and lambda represents the rate-distortion cost coefficient of the code rate; w is a ₁ And w ₂ The two adjustment coefficients can be obtained by pre-testing; RD-RB (mode-j) represents a first price estimation unit for other image blocksAnd RD-rec (mode-j) represents a second cost estimate unit for the reference pixel.

Wherein RD-RB (mode-j) ═ { K-R (| SATD-R) _{Reconstruction} －SATD-R _Original |)+K-B*(|SATD-B _{Reconstruction} －SATD-B _Original |) }, namely the first price estimating unit for other image blocks comprises two parts, namely the first price estimating unit for the right image block and the first price estimating unit for the lower image block. K-R and K-B represent weights for the rightmost pixel column and the bottommost pixel row, respectively, | SATD-R _{Reconstruction} And S ATD-R _Original I respectively represents the first residual information and the second residual information of the right image block, | SATD-B _{Reconstruction} －SATD-B _Original And | respectively represents first residual information and second residual information of the lower image block.

Wherein RD-rec (mode-j) ═ SAD (Pix-right) _Original －Pix-right _{Reconstruction} )+SAD(Pix-bottow _Original －Pix-bottow _{Reconstruction} ) That is, the second-price estimating unit for the reference pixel includes the second-price estimating unit for the rightmost pixel column and the second-price estimating unit for the lowermost pixel row. Pix-right _Original And Pix-right _{Reconstruction} Respectively representing the original pixel value and the reconstructed pixel value of the rightmost pixel column, Pix-bottom _Original －Pix-bottow _{Reconstruction} Respectively representing the original pixel value and the reconstructed pixel value of the lowest pixel row, and sad (sum of Absolute difference) representing the sum of Absolute errors of each pixel point.

And respectively estimating the rate distortion cost values of the target image block in various candidate intra-frame prediction modes through the rate distortion cost estimation model to obtain the rate distortion cost values of the target image block in the candidate intra-frame prediction modes, and determining the candidate intra-frame prediction mode corresponding to the rate distortion cost value with the minimum value in the rate distortion cost values as the target intra-frame prediction mode.

In this embodiment, a first cost estimation unit for other image blocks and a second cost estimation unit for a reference pixel in a target image block are introduced into the rate-distortion estimation model, and the influence of an intra-frame prediction mode on the target image block and on other image blocks is fully considered, so that the determined target intra-frame prediction mode is the optimal intra-frame prediction mode of the target image block, the accuracy of an intra-frame prediction result of a video frame can be improved, and the quality of video coding is improved.

In another exemplary embodiment, as shown in fig. 4, a flow chart of another intra prediction method according to an exemplary embodiment is shown, in which the method includes the following steps:

step S410, determining reference pixels of a target image block in a video frame, and determining rate distortion cost values of the target image block in multiple candidate intra-frame prediction modes through a rate distortion cost estimation model;

step S420, determining the candidate intra-frame prediction mode corresponding to the rate distortion cost value with the minimum value in the rate distortion cost values as the target intra-frame prediction mode of the target image block;

step S430A, acquiring a reconstructed pixel value of the target image block in the target intra-frame prediction mode, performing intra-frame prediction on other image blocks through the reconstructed pixel value of the target image block, and acquiring first residual error information of the other image blocks based on a first predicted pixel value obtained through prediction and original pixel values of the other image blocks;

step S430B, performing intra-frame prediction on other image blocks through the original pixel values of the target image block, and obtaining second residual error information of the other image blocks based on the second predicted pixel values obtained by prediction and the original pixel values of the other image blocks;

step S440, carrying out subtraction on the first residual error information and the second residual error information to obtain a first initial cost value for other image blocks; weighting the first initial cost value according to the weight of the reference pixel to obtain a first generation value for other image blocks;

step S450, the difference is made between the original pixel value of the reference pixel and the reconstructed pixel value of the reference pixel to obtain a second generation value aiming at the reference pixel;

step S460, determining a compensation mode for the reference pixel in the target image block based on the first cost value for the other image blocks and the second cost value for the reference pixel;

step S470, determining a compensation value calculation relation for the reference pixel according to the compensation mode, and determining a compensation pixel value for the reference pixel based on the compensation value calculation relation;

in step S480, the pixel value of the reference pixel is compensated based on the compensation pixel value.

In the intra prediction method provided in this embodiment, intra prediction is performed on other image blocks through an original pixel value and a reconstructed pixel value of a target image block, so as to obtain first residual information and second residual information, a compensation mode for a reference pixel in the target image block is determined according to the first residual information, the second residual information, and the original pixel value and the reconstructed pixel value of the reference pixel, and then the pixel value of the reference pixel is compensated according to the compensation mode, so that the pixel value of the reference pixel is closer to the original pixel value, distortion of the reference pixel is reduced, and thus accuracy of an intra prediction result is improved.

In an exemplary embodiment, to facilitate an understanding of the disclosed embodiments by those of ordinary skill in the art, reference will now be made to the specific examples illustrated in the drawings. Referring to fig. 5, an overall flow chart of a method for compensating intra prediction and coding based on mode dependency and reference pixels in an application example is shown.

Before encoding:

(1) and performing texture analysis on the video frame. Specifically, a video frame is divided into NxN (N may be any value, for example, 64, 32, 16, etc.) image blocks, and texture detection is performed on each image block to obtain a texture type (or texture direction) of each image block.

(2) And determining the weight of the reference pixel of each image block according to the texture direction of other image blocks influenced by each image block. Specifically, the video frame is traversed from top left to bottom right in raster order, determining for each image block the effect on subsequent coding blocks. Since the rightmost pixel column of the target image block affects the prediction result of the right image block of the target image block, and the bottommost pixel row of the target image block affects the prediction result of the lower image block of the target image block, the weight of the rightmost pixel column can be determined according to the estimated model K-R-clip 3(0, X, abs (cot-M-R)), and the weight of the bottommost pixel row can be determined according to the estimated model K-B-clip 3(0, X, abs (cot-M-B)).

In the encoding:

(3) a compensation mode for reference pixels in each image block is determined based on a rate distortion optimization method (RDO). Specifically, for each image block, rate distortion cost values of the image block in a plurality of candidate intra-frame prediction modes are estimated through a rate distortion cost estimation model, so that a candidate intra-frame prediction mode corresponding to the rate distortion cost value with the smallest value in the rate distortion cost values is obtained, and the candidate intra-frame prediction mode is determined as the optimal intra-frame prediction mode (namely, the target intra-frame prediction mode) of the image block. Wherein the rate-distortion cost value for each image block includes a first generation value for the right image block, a first generation value for the lower image block, a second generation value for the rightmost pixel column, and a second generation value for the bottommost pixel row.

The method for determining the compensation Mode Use _ Mode includes:

a: user _ Mode is equal to 1, the compensation Mode is set as the right side compensation Mode, and the conditions are as follows: the first cost value of the right image block is greater than the first threshold, and the second cost value of the rightmost pixel column is greater than the second threshold.

B: use _ Mode is 2, and the compensation Mode is set as the lower compensation Mode, with the conditions: the first cost value of the lower image block is greater than a first threshold and the second cost value of the lowermost pixel row is greater than a second threshold.

C: user _ Mode is 3, the compensation Mode is set to the double-side compensation Mode, and the conditions are as follows: the conditions of Use _ Mode ═ 1 and Use _ Mode ═ 2 are satisfied.

D: use _ Mode is equal to 0, and the compensation Mode is set to be uncompensated, provided that:

d1, the residual information of the image block predicted by the original pixel values is larger than the third threshold, or,

d2, the second cost value of the rightmost pixel column is less than the second threshold, or,

d3, the second cost value of the lowermost pixel row is less than the second threshold, or,

d4, the sum of the first cost value of the right image block and the second cost value of the lower image block is less than a fourth threshold.

D5, and all other cases where the compensation condition is not satisfied.

(4) And performing coding compensation on the reference pixel based on the determined compensation mode. Specifically, the flag context model may be used for entropy coding, and when Use _ Mode >1, the compensation pixel value is determined according to a compensation value calculation relation under different modes, so as to perform coding compensation, where the coding Mode may refer to encoder residual entropy coding, and as shown in fig. 6, the order of the compensation pixel values is from top right → bottom left.

At the decoding end:

(5) and the decoder decodes the intra-frame prediction mode according to the coding standard to obtain a reconstructed image block of each image block.

(6) The compensation mode/compensation pixel values are decoded.

When Use _ Mode is 0, decoding is performed by skipping the compensation Mode.

When Use _ Mode >1, the compensation Mode for the image block is decoded, as well as the compensated pixel values for the rightmost pixel column and the bottommost pixel row.

(7) The reference pixel is compensated. And adding corresponding compensation pixel values to the rightmost pixel column and the bottommost pixel row of the reconstructed image block of each image block.

The intra-frame prediction rate distortion optimization method provided by the embodiment selectively reduces the distortion of the reference pixel by combining the characteristics of video content, thereby improving the accuracy of intra-frame prediction and improving the compression quality and efficiency of an encoder.

It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.

It is understood that the same/similar parts between the embodiments of the method described above in this specification can be referred to each other, and each embodiment focuses on the differences from the other embodiments, and it is sufficient that the relevant points are referred to the descriptions of the other method embodiments.

Based on the same inventive concept, the embodiment of the present disclosure further provides an intra prediction apparatus for implementing the above-mentioned intra prediction method.

Fig. 7 is a block diagram illustrating a structure of an intra prediction apparatus according to an exemplary embodiment. Referring to fig. 7, the apparatus includes: a prediction mode determination unit 710, a first residual acquisition unit 720, a second residual acquisition unit 730, a compensation mode determination unit 740, and an intra prediction unit 750, wherein,

a prediction mode determination unit 710 configured to perform determining a target intra prediction mode and a reference pixel for a target image block in a video frame; the target image block is any one of a plurality of image blocks obtained after a video frame is divided, the reference pixel represents a pixel which affects the intra-frame prediction results of other image blocks in the target image block, and the other image blocks represent image blocks which are adjacent to the target image block and the pixel values of which are affected by the target image block;

a first residual obtaining unit 720, configured to perform obtaining of a reconstructed pixel value of the target image block in the target intra prediction mode, and obtain first residual information of other image blocks through prediction of the reconstructed pixel value of the target image block;

a second residual obtaining unit 730 configured to perform second residual information of the other image blocks obtained by predicting the original pixel values of the target image block;

a compensation mode determination unit 740 configured to perform determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel;

and an intra prediction unit 750 configured to perform compensation of the pixel value of the reference pixel according to the compensation mode, and perform intra prediction on other image blocks affected by the target image block based on the compensated reference pixel value.

In an exemplary embodiment, the first residual obtaining unit 720 is specifically configured to perform intra prediction on other image blocks by using reconstructed pixel values of the target image block, so as to obtain first predicted pixel values of the other image blocks; and obtaining first residual error information of other image blocks based on the first predicted pixel value and the original pixel values of the other image blocks.

In an exemplary embodiment, the second residual obtaining unit 730 is specifically configured to perform intra prediction on other image blocks through the original pixel value of the target image block to obtain second predicted pixel values of the other image blocks; and obtaining second residual error information of other image blocks based on the second predicted pixel values and the original pixel values of the other image blocks.

In an exemplary embodiment, the compensation mode determining unit 740 is specifically configured to perform a subtraction of the first residual information and the second residual information to obtain a first cost value for the other image blocks, and a subtraction of the original pixel value of the reference pixel and the reconstructed pixel value of the reference pixel to obtain a second cost value for the reference pixel; a compensation mode for a reference pixel in the target image block is determined based on the first cost value for the other image blocks and the second cost value for the reference pixel.

In an exemplary embodiment, the reference pixels include a rightmost pixel column and a bottommost pixel row of the target image block, and the other image blocks include a right image block of the target image block and a lower image block of the target image block; a compensation mode determination unit 740 further configured to perform acquiring a first generation value of the right image block, a first generation value of the lower image block, a second generation value of the rightmost pixel column, and a second generation value of the lowermost pixel row; determining the compensation mode aiming at the reference pixel as a right side compensation mode under the condition that the first cost value of the right side image block is larger than a first threshold value and the second cost value of the rightmost pixel column is larger than a second threshold value; determining a compensation mode for the reference pixel as a lower side compensation mode in case that the first cost value of the lower image block is greater than a first threshold and the second cost value of the lowermost pixel row is greater than a second threshold; determining that the compensation mode aiming at the reference pixel is a bilateral compensation mode under the condition that the first cost values of the right image block and the lower image block are both larger than a first threshold value, and the second cost values of the rightmost pixel column and the bottommost pixel row are both larger than a second threshold value; the double-sided compensation mode includes a right-side compensation mode and a lower-side compensation mode.

In an exemplary embodiment, the compensation mode determining unit 740 is further configured to perform subtraction on the first residual information and the second residual information to obtain a first initial cost value for the other image blocks; acquiring a weight value aiming at the reference pixel; the weight represents the influence degree of the reference pixel on the intra-frame prediction results of other image blocks; and weighting the first initial cost value through the weight value to obtain a first generation value for other image blocks.

In an exemplary embodiment, the compensation mode determining unit 740 is further configured to perform a third residual information of the target image block predicted by the original pixel values of the video frame; and under the condition that the first residual error information, the second residual error information, the third residual error information, the original pixel value of the reference pixel and the reconstructed pixel value of the reference pixel are determined not to meet the preset compensation condition, determining that the compensation mode aiming at the reference pixel is not compensated.

In an exemplary embodiment, the intra prediction unit 750 is specifically configured to perform determining a compensation value calculation relation for the reference pixel according to the compensation mode, and determining a compensation pixel value for the reference pixel based on the compensation value calculation relation; the pixel value of the reference pixel is compensated based on the compensation pixel value.

In an exemplary embodiment, the prediction mode determining unit 710 is specifically configured to perform determining, by a rate-distortion cost estimation model, rate-distortion cost values of the target image block in a plurality of candidate intra prediction modes; the rate-distortion estimation model comprises a first cost estimation unit for other image blocks and a second cost estimation unit for reference pixels in the target image block; and determining the candidate intra-frame prediction mode corresponding to the rate distortion cost value with the minimum value in the rate distortion cost values as the target intra-frame prediction mode.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Fig. 8 is a block diagram illustrating an electronic device 800 for implementing a method for intra prediction according to an example embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a gaming console, a tablet device, a medical device, a fitness device, a personal digital assistant, and so forth.

Referring to fig. 8, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile storage devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or graphene memory.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or components of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration/deceleration of the device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 804 comprising instructions, executable by the processor 820 of the electronic device 800 to perform the above-described method is also provided. For example, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided that includes instructions executable by the processor 820 of the electronic device 800 to perform the above-described method.

It should be noted that the descriptions of the above-mentioned apparatus, the electronic device, the computer-readable storage medium, the computer program product, and the like according to the method embodiments may also include other embodiments, and specific implementations may refer to the descriptions of the related method embodiments, which are not described in detail herein.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An intra prediction method, comprising:

determining a target intra prediction mode and reference pixels for a target image block in a video frame; the target image block is any one of a plurality of image blocks obtained after the video frame is divided, the reference pixel represents a pixel which affects intra-frame prediction results of other image blocks in the target image block, and the other image blocks represent image blocks which are adjacent to the target image block and have pixel values affected by the target image block;

predicting through the original pixel value of the target image block to obtain second residual error information of other image blocks;

2. The method according to claim 1, wherein said deriving first residual information of the other image blocks by predicting reconstructed pixel values of the target image block comprises:

3. The method according to claim 1, wherein obtaining second residual information of the other image blocks through prediction of original pixel values of the target image block comprises:

4. The method of claim 1, wherein determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel comprises:

5. The method of claim 4, wherein the reference pixels comprise a rightmost pixel column and a bottommost pixel row of the target image block, and the other image blocks comprise a right image block of the target image block and a lower image block of the target image block;

6. The method of claim 4, wherein the subtracting the first residual information and the second residual information to obtain a first cost value for the other image blocks comprises:

7. The method of claim 6, wherein obtaining the weight values for the reference pixels comprises:

8. The method of claim 1, wherein determining a compensation mode for a reference pixel in the target image block based on the first residual information, the second residual information, an original pixel value of the reference pixel, and a reconstructed pixel value of the reference pixel, further comprises:

predicting to obtain third residual information of the target image block through an original pixel value of the video frame;

9. The method of claim 1, wherein compensating the pixel value of the reference pixel according to the compensation pattern comprises:

10. The method of claim 1, wherein determining the target intra prediction mode for the target image block in the video frame comprises:

11. An intra prediction apparatus, comprising:

a prediction mode determination unit configured to perform determination of a target intra prediction mode and reference pixels for a target image block in a video frame; the target image block is any one of a plurality of image blocks obtained after the video frame is divided, the reference pixel represents a pixel which affects intra-frame prediction results of other image blocks in the target image block, and the other image blocks represent image blocks which are adjacent to the target image block and have pixel values affected by the target image block;

12. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the intra prediction method of any of claims 1 to 10.

13. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the intra prediction method of any of claims 1-10.

14. A computer program product comprising instructions therein, which when executed by a processor of an electronic device, enable the electronic device to perform the intra prediction method of any of claims 1 to 10.