CN112243129A

CN112243129A - Video data processing method and device, computer equipment and storage medium

Info

Publication number: CN112243129A
Application number: CN202011104871.XA
Authority: CN
Inventors: 黄博; 闻兴
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2020-10-15
Filing date: 2020-10-15
Publication date: 2021-01-19
Anticipated expiration: 2040-10-15
Also published as: CN112243129B

Abstract

The disclosure relates to a video data processing method and device, computer equipment and a storage medium, and belongs to the technical field of audio and video. According to the method and the device, after one video frame is coded, the quantization parameter applied to the subsequent video frame coding is adjusted based on the actual code rate and the actual distortion information of the video frame, so that the code rate of the subsequent video frame after coding is controlled, the total code rate of the video frame after coding can be controlled in a reasonable range, the actual distortion information of the video frame is considered in the quantization parameter adjusting process, and the total code rate of the video can meet the code rate requirement under the condition that the quality of the coded video is ensured.

Description

Video data processing method and device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of audio and video technologies, and in particular, to a data processing method and apparatus, a computer device, and a storage medium.

Background

The code rate control can embody the adaptability of the encoder in practical application, the relationship between the code rate and the video quality needs to be balanced through the code rate control in the process that the encoder encodes the video frame, and the code rate of the video meets the code rate requirement of a video playing platform under the condition of ensuring the video quality.

At present, the SATD-R model is usually applied to adjust the code rate of each video frame, so that the overall code rate of the video meets the code rate requirement of a video playing platform. For example, when a video is coded, the SATD-R model learns the code rate and the quantization parameter of the coded video frame to predict the code rate required when the subsequent video frame is coded, and adjusts the quantization parameter applied when the subsequent video frame is coded. However, in the above video data processing process, for the first frames in the video, the SATD-R model learns less data, and the model precision is extremely low, which results in inaccurate code rate prediction of the SATD-R model for the first frames, thereby affecting code rate allocation for subsequent video frames in the video and further affecting the quality of the encoded video. Therefore, in the process of processing video data, how to adjust the quantization parameter of each video frame to control the code rate of each video frame so as to ensure that the overall code rate of the video meets the requirement and ensure the video quality is an important research direction.

Disclosure of Invention

The present disclosure provides a video data processing method, apparatus, computer device, and storage medium, which can control a bit rate of a video while ensuring video quality. The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a video data processing method, including

Acquiring a plurality of video frames to be coded and a quantization parameter corresponding to each video frame;

for any video frame in the plurality of video frames, coding the any video frame based on the quantization parameter corresponding to the any video frame to obtain the actual code rate and the actual distortion information corresponding to the any video frame;

and adjusting the quantization parameter corresponding to the next video frame based on the actual code rate corresponding to any video frame and the matching relation between the actual distortion information and the first reference condition, and encoding the next video frame based on the adjusted quantization parameter.

In one possible implementation, the obtaining a plurality of video frames to be encoded and a quantization parameter corresponding to each video frame includes:

acquiring the plurality of video frames to be coded and an initial quantization parameter;

determining a prediction code rate and prediction distortion information corresponding to each video frame based on the initial quantization parameter and the pixel information corresponding to each video frame;

and adjusting the initial quantization parameter corresponding to each video frame based on the matching relationship between the prediction code rate and the prediction distortion information and a second reference condition to obtain the quantization parameter corresponding to each video frame.

In one possible implementation, the determining, based on the initial quantization parameter and the pixel information corresponding to each video frame, a prediction code rate and prediction distortion information corresponding to each video frame includes:

based on the correlation between the pixel points in each video frame, performing intra-frame prediction and inter-frame prediction on each video frame to obtain predicted image data corresponding to each video frame;

respectively subtracting the video frames from the corresponding predicted image data to obtain predicted residual data corresponding to the video frames;

and determining the prediction code rate and the prediction distortion information corresponding to each video frame based on the prediction residual data corresponding to each video frame and the initial quantization parameter.

In one possible implementation, the determining, based on the prediction residual data and the initial quantization parameter corresponding to each video frame, a prediction code rate and prediction distortion information corresponding to each video frame includes:

determining a plurality of residual coefficients corresponding to each video frame based on the predicted residual data corresponding to each video frame;

acquiring a reference model, wherein the reference model is used for predicting code rate and distortion information of a video frame based on a quantization parameter and a residual coefficient;

and respectively inputting the initial quantization parameter corresponding to each video frame and the residual error coefficients into the reference model, and outputting the prediction code rate and the prediction distortion information corresponding to each video frame by the reference model.

In a possible implementation manner, the adjusting the initial quantization parameter corresponding to each video frame based on the matching relationship between the prediction code rate and the prediction distortion information and a second reference condition to obtain the quantization parameter corresponding to each video frame includes:

in response to the sum of the predicted code rates being greater than a first code rate threshold, increasing the value of the initial quantization parameter corresponding to each video frame, and determining the predicted code rate and the predicted distortion information corresponding to each video frame based on the adjusted initial quantization parameter until the sum of the predicted code rates corresponding to each video frame is less than the first code rate threshold;

determining the initial quantization parameter currently corresponding to each video frame as the quantization parameter in response to the fact that the sum of the predicted code rates is smaller than the first code rate threshold and the fluctuation amplitude of each piece of the predicted distortion information is smaller than a fluctuation threshold;

and in response to that the sum of the prediction code rates is smaller than the first code rate threshold and the fluctuation amplitude of each prediction distortion information is larger than the fluctuation threshold, reducing the initial quantization parameter of the video frame corresponding to the maximum prediction distortion information, and increasing the initial quantization parameter of the video frame corresponding to the minimum prediction distortion information to obtain the quantization parameter corresponding to each video frame.

In a possible implementation manner, the encoding any video frame based on the quantization parameter corresponding to the any video frame to obtain the actual bitrate and the actual distortion information corresponding to the any video frame includes:

coding each image block in any video frame based on the quantization parameter corresponding to the video frame to obtain an intermediate code rate and intermediate distortion information corresponding to each image block;

and determining the actual code rate and the actual distortion information corresponding to any video frame based on the intermediate code rate and the intermediate distortion information corresponding to each image block.

In a possible implementation manner, the encoding each image block in any video frame based on the quantization parameter corresponding to the any video frame to obtain an intermediate code rate and intermediate distortion information corresponding to each image block includes:

for a first image block in any video frame, coding the first image block based on a quantization parameter corresponding to the any video frame to obtain an intermediate code rate and intermediate distortion information corresponding to the first image block;

for any image block except the first image block in any video frame, acquiring an intermediate code rate and intermediate distortion information corresponding to a previous image block, adjusting a quantization parameter currently corresponding to any video frame based on the intermediate code rate and the intermediate distortion information corresponding to the previous image block, and encoding any image block based on the adjusted quantization parameter.

In a possible implementation manner, the adjusting the quantization parameter currently corresponding to any video frame based on the intermediate bitrate and the intermediate distortion information corresponding to the previous image block includes:

responding to the condition that the middle code rate corresponding to the previous image block is larger than a second code rate threshold value, and adding one to the numerical value of the quantization parameter to obtain the adjusted quantization parameter;

and in response to that the intermediate code rate corresponding to the previous image block is smaller than the second code rate threshold, adjusting the quantization parameter based on the size relationship between the intermediate distortion information and the intermediate distortion threshold to obtain the adjusted quantization parameter.

In one possible implementation manner, the adjusting the quantization parameter based on the magnitude relationship between the intermediate distortion information and an intermediate distortion threshold to obtain the adjusted quantization parameter includes:

responding to the numerical value of the intermediate distortion information being larger than the intermediate distortion threshold value, and acquiring a first error value between the intermediate code rate and the second code rate threshold value;

and in response to the first error value being greater than a first error threshold, subtracting one from the value of the quantization parameter to obtain the adjusted quantization parameter.

In a possible implementation manner, the adjusting the quantization parameter corresponding to the next video frame based on the actual bitrate and the matching relationship between the actual distortion information and the first reference condition corresponding to the any video frame includes:

in response to the actual code rate corresponding to the any video frame being greater than a third code rate threshold, adding one to the quantization parameter corresponding to the next video frame;

and in response to that the actual code rate corresponding to the any video frame is smaller than the third code rate threshold, adjusting the quantization parameter corresponding to the next video frame based on the size relationship between the actual distortion information corresponding to the any video frame and the actual distortion threshold.

In a possible implementation manner, the adjusting the quantization parameter corresponding to the next video frame based on a size relationship between the actual distortion information corresponding to the any video frame and an actual distortion threshold includes:

responding to that the actual distortion information corresponding to any video frame is larger than the actual distortion threshold, and acquiring a second error value between the actual code rate of any video frame and a third code rate threshold;

in response to the second error value being greater than a second error threshold, one is decreased for the quantization parameter corresponding to the next video frame.

In one possible implementation, the method is applied to a video encoder.

According to a second aspect of the embodiments of the present disclosure, there is provided a video data processing apparatus comprising:

the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire a plurality of video frames to be coded and a quantization parameter corresponding to each video frame;

a second obtaining module, configured to encode, for any video frame in the plurality of video frames, the any video frame based on a quantization parameter corresponding to the any video frame, so as to obtain an actual bitrate and actual distortion information corresponding to the any video frame;

and the adjusting module is configured to adjust a quantization parameter corresponding to a next video frame based on the actual code rate corresponding to any one video frame and the matching relationship between the actual distortion information and a first reference condition, and encode the next video frame based on the adjusted quantization parameter.

In one possible implementation, the first obtaining module includes:

an obtaining sub-module configured to obtain the plurality of video frames to be encoded and an initial quantization parameter;

a first determining sub-module, configured to determine a prediction code rate and prediction distortion information corresponding to each video frame based on the initial quantization parameter and pixel information corresponding to each video frame;

and the adjusting sub-module is configured to adjust the initial quantization parameter corresponding to each video frame based on the matching relationship between the prediction code rate and the prediction distortion information and a second reference condition, so as to obtain the quantization parameter corresponding to each video frame.

In one possible implementation, the first determining sub-module includes:

the prediction unit is configured to perform intra-frame prediction and inter-frame prediction on each video frame based on the correlation between the pixel points in each video frame to obtain predicted image data corresponding to each video frame;

the acquisition unit is configured to respectively perform difference on each video frame and the corresponding predicted image data to obtain predicted residual data corresponding to each video frame;

a determining unit configured to determine a prediction code rate and prediction distortion information corresponding to each video frame based on the prediction residual data corresponding to each video frame and the initial quantization parameter.

In one possible implementation, the prediction unit is configured to:

In one possible implementation, the adjustment submodule is configured to:

In one possible implementation, the second obtaining module includes:

the coding sub-module is configured to code each image block in any video frame based on the quantization parameter corresponding to the any video frame to obtain an intermediate code rate and intermediate distortion information corresponding to each image block;

a second determining sub-module configured to determine the actual bitrate and the actual distortion information corresponding to the any video frame based on the intermediate bitrate and the intermediate distortion information corresponding to the respective image blocks.

In one possible implementation, the encoding submodule includes:

a first encoding unit, configured to encode, for a first image block in the any video frame, the first image block based on a quantization parameter corresponding to the any video frame, so as to obtain an intermediate code rate and intermediate distortion information corresponding to the first image block;

the second coding unit is configured to acquire an intermediate code rate and intermediate distortion information corresponding to a previous image block for any image block, except for the first image block, in any video frame, adjust a quantization parameter currently corresponding to the any video frame based on the intermediate code rate and the intermediate distortion information corresponding to the previous image block, and code the any image block based on the adjusted quantization parameter.

In one possible implementation, the second encoding unit includes:

a first adjusting subunit, configured to add one to the numerical value of the quantization parameter in response to that the intermediate code rate corresponding to the previous image block is greater than a second code rate threshold, so as to obtain the adjusted quantization parameter;

and the second adjusting subunit is configured to adjust the quantization parameter based on a size relationship between the intermediate distortion information and an intermediate distortion threshold value in response to that the intermediate code rate corresponding to the previous image block is smaller than the second code rate threshold value, so as to obtain the adjusted quantization parameter.

In one possible implementation, the second adjusting subunit is configured to:

In one possible implementation, the adjustment module includes:

a first adjusting submodule configured to add one to the quantization parameter corresponding to the next video frame in response to the actual bitrate corresponding to the any video frame being greater than a third bitrate threshold;

and a second adjusting sub-module, configured to adjust the quantization parameter corresponding to the next video frame based on a size relationship between actual distortion information corresponding to the any video frame and an actual distortion threshold value in response to the actual code rate corresponding to the any video frame being smaller than the third code rate threshold value.

In one possible implementation, the second adjusting submodule is configured to:

In one possible implementation, the apparatus is applied to a video encoder.

According to a third aspect of embodiments of the present disclosure, there is provided a computer device comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement any of the video data processing methods as described above.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, in which instructions, when executed by a processor of a computer device, enable the computer device to perform any one of the video data processing methods described above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising executable instructions that, when executed by a processor of a computer device, enable the computer device to perform any of the video data processing methods as described above.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

after one video frame is coded, the quantization parameter applied to the subsequent video frame coding is adjusted based on the actual code rate and the actual distortion information of the video frame, so that the code rate of the subsequent video frame after coding is controlled, the total code rate of the video frame after coding can be controlled in a reasonable range, and the actual distortion information of the video frame is considered in the quantization parameter adjusting process, so that the total code rate of the video can meet the code rate requirement under the condition of ensuring the quality of the coded video.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a flow diagram illustrating a method of video data processing in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a method of video data processing in accordance with an exemplary embodiment;

FIG. 3 is a diagram illustrating a method for adjusting quantization parameters at a video frame level according to an exemplary embodiment;

fig. 4 is a diagram illustrating a quantization parameter adjustment method at an image block level according to an exemplary embodiment;

FIG. 5 is a flow chart illustrating a rate control method in accordance with an example embodiment;

FIG. 6 is a block diagram illustrating a video data processing apparatus according to an exemplary embodiment;

FIG. 7 is a block diagram illustrating a computer device in accordance with an example embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

Fig. 1 is a flowchart illustrating a video data processing method according to an exemplary embodiment, which is used in a computer device, as shown in fig. 1, and includes the following steps.

In step 101, a plurality of video frames to be encoded and a quantization parameter corresponding to each video frame are obtained.

In step 102, for any video frame of the plurality of video frames, the any video frame is encoded based on the quantization parameter corresponding to the any video frame, so as to obtain the actual bitrate and the actual distortion information corresponding to the any video frame.

In step 103, a quantization parameter corresponding to a next video frame is adjusted based on the actual bitrate corresponding to the any video frame and the matching relationship between the actual distortion information and a first reference condition, and the next video frame is encoded based on the adjusted quantization parameter.

According to the technical scheme provided by the embodiment of the disclosure, after one video frame is coded, the quantization parameter applied by the subsequent video frame coding is adjusted based on the actual code rate and the actual distortion information of the video frame, so that the code rate of the subsequent video frame after coding is controlled, the total code rate of the video frame after coding can be controlled in a reasonable range, and the actual distortion information of the video frame is considered in the quantization parameter adjusting process, so that the total code rate of the video can meet the code rate requirement under the condition of ensuring the quality of the coded video.

In one possible implementation, the method is applied to a video encoder.

The above embodiments are only a brief introduction to the embodiments of the present disclosure and various possible implementation manners, and the technical solution provided by the embodiments of the present disclosure may be applied to a video encoder, for example, any type of computer device such as a terminal or a server is equipped with a video encoder, and in combination with the technical solution provided by the embodiments of the present disclosure, when the video encoder in the computer device encodes each video frame, the video encoder may predict the code rate and distortion of the video frame based on the specified quantization parameter, adjust the quantization parameter based on the predicted data, and then apply the quantization parameter to actual encoding, or after the current video frame is encoded, adjust the quantization parameter applied when the next video frame is encoded based on the actual code rate and distortion of the current video frame, so as to achieve the encoding situation according to the encoded video frame, the effect of controlling the code rate of the subsequent video frame enables the total code rate corresponding to the video to meet the requirement under the condition of ensuring the video quality. Referring to fig. 2, fig. 2 is a flowchart illustrating a video data processing method according to an exemplary embodiment, and the video data processing method is described below with reference to fig. 2, where a computer device is taken as an execution subject, and the method specifically includes the following steps:

in step 201, a computer device obtains a plurality of video frames to be encoded and an initial quantization parameter.

Wherein the quantization parameter is used for encoding the video frame. For example, one quantization parameter corresponds to one quantization step, and the computer device obtains the quantization step indicated by this quantization parameter from the correspondence table of quantization parameters and quantization steps based on the quantization parameter, and quantizes and encodes the video frame based on the quantization step. In the embodiment of the present disclosure, the initial quantization parameter is set by a developer, which is not limited by the embodiment of the present disclosure.

In one possible implementation, the computer device obtains a plurality of video frames and the initial quantization parameter from a target video to be encoded in response to an encoding instruction. The target video may be a video stored in the computer device, or may also be a video acquired by the computer device in real time, which is not limited in this disclosure. Taking the example that the target video is a video stored in the computer device, the computer device acquires a plurality of consecutive video frames from the first video frame of the target video in response to the encoding instruction. It should be noted that the number of video frames acquired by the computer device at a time is set by a developer, and the embodiment of the disclosure does not limit this.

In step 202, the computer device determines a prediction code rate and prediction distortion information corresponding to each video frame based on the initial quantization parameter and the pixel information corresponding to each video frame.

In the embodiment of the disclosure, after obtaining a plurality of video frames to be encoded, a computer device first predicts a code rate required by each video frame during encoding and a distortion condition after encoding, and obtains a prediction code rate and prediction distortion information corresponding to each video frame. In one possible implementation, the process may include the steps of:

step one, the computer equipment carries out intra-frame prediction and inter-frame prediction on each video frame based on the correlation between pixel points in each video frame to obtain predicted image data corresponding to each video frame.

Taking the example of obtaining the predicted image data corresponding to one video frame, in one possible implementation manner, the computer device first divides the video frame into image blocks that do not overlap with each other. For example, the computer device may perform 1/2 downsampling on a video frame first, and perform image block division on the downsampled video frame, and of course, may also use a quadtree division manner to divide a video frame into image blocks with different sizes and without overlapping each other. And based on a predictive coding mode, the computer equipment removes redundant information by utilizing the spatial correlation and the temporal correlation of the video content to obtain a predictive image block corresponding to each image block so as to obtain predictive image data corresponding to the video frame. The prediction coding includes intra prediction and inter prediction in the hevc standard, and the embodiment of the present disclosure does not limit which method is specifically adopted for prediction coding.

And step two, the computer equipment respectively makes a difference between each video frame and the corresponding predicted image data to obtain the predicted residual data corresponding to each video frame.

The prediction residual data corresponding to one video frame comprises residual block data corresponding to each image block in the one video frame. In a possible implementation manner, for a video frame, the computer device performs a difference between each image block in the video frame and the corresponding predicted image block to obtain residual block data corresponding to each image block, and uses the residual block data corresponding to each image block as predicted residual data corresponding to the video frame.

It should be noted that, the description of the method for acquiring the prediction residual data in the first step and the second step is only an exemplary description, and the embodiment of the present disclosure does not limit which method is specifically used to acquire the prediction residual data.

And step three, the computer equipment determines the prediction code rate and the prediction distortion information corresponding to each video frame based on the prediction residual data corresponding to each video frame and the initial quantization parameter.

In one possible implementation manner, first, the computer device determines a transformation method corresponding to each residual block data based on the size of the residual block data corresponding to the image block in each video frame and a prediction mode adopted in the video encoding process. For example, in the hevc coding standard, 4 × 4 residual block data is transformed by DST (Discrete Sine Transform), and residual block data of other sizes is transformed by DCT (Discrete Cosine Transform). After the computer equipment applies the corresponding transformation method to transform the residual block data, the residual coefficient can be obtained. In the embodiment of the present disclosure, DCT or DST transform is not required to be performed on each residual block data at this stage, and only the transformed residual coefficients need to be estimated. In one possible implementation, the residual coefficients may be estimated by the following equations (1) and (2) instead of the DCT and DST transform processes, respectively. The size of each transformation matrix in formula (1) may be any one of 64 × 64, 32 × 32, 16 × 16, 8 × 8, and 4 × 4, and the size of each matrix in formula (2) is 4 × 4. The equations (1) and (2) can be specifically expressed as:

C1＝HXH’ (1)

C2＝BXB’ (2)

wherein, X represents residual block data corresponding to any image block in the video frame; c1 denotes the estimated DCT transformed residual coefficients; h denotes a sequential Ordered Walsh-Hadamard Matrix (Walsh-Hadamard transform Matrix), and H' denotes a transpose of the Matrix H; c2 denotes the estimated DST transformed residual coefficients; b denotes a simplified transformation matrix that approximates the DST characteristics, which can be expressed specifically by the following formula (3), with B' representing the transpose of matrix B.

It should be noted that the above description of estimating the residual coefficients after DCT and DST transformation is only an exemplary description, and the embodiment of the present disclosure does not limit which method is specifically used to estimate the residual coefficients after DCT and DST transformation. In the embodiment of the disclosure, a low-complexity residual coefficient estimation method is adopted, and the matrix H, the matrix B and residual block data corresponding to each predicted image block are directly applied to estimate residual coefficients after DCT and DST transformation, so that subsequent reconstruction processes, such as DCT or DST transformation, and subsequent quantization and entropy coding processes can be omitted, and a code rate can be estimated with lower complexity.

The computer device then acquires a reference model. The reference model is used for predicting the code rate and distortion information corresponding to the coded video frame based on the quantization parameter and the estimated residual error coefficient. In the embodiment of the disclosure, the reference model is a code rate-distortion model based on video content characteristics, and the reference model fully considers the content characteristics of the video frame when predicting code rate and distortion information.

And finally, the computer equipment respectively inputs the initial quantization parameter corresponding to each video frame and the estimated residual coefficient into the reference model, and the reference model outputs the prediction code rate and the prediction distortion information corresponding to each video frame. In one possible implementation, the corresponding predicted bitrate of a video frame can be expressed as the following formula (4):

wherein,

representing the corresponding predicted code rate of the video frame r; α, β, and γ represent model parameters; CG denotes a divided 4 x 4 size non-intersecting block in the image block; xi_iRepresenting the CG coefficient characteristics, i representing the CG number, ξ_iCan be determined based on the following equation (5):

where Δ denotes a quantization step size, S denotes a size of an image block, c_iRepresentation estimationAmong the calculated residual coefficients, the coefficient at the position (x, y); TH denotes a quantization dead zone threshold.

In one possible implementation, the video frame corresponding prediction distortion information may be expressed as the following formula (6):

wherein,

representing prediction distortion information;

representing coefficients within a quantization dead zone, i.e. | c_i|<Distortion information of TH whose value is equal to the sum of squares of respective coefficients within the quantization dead zone;

representing coefficients outside the quantization dead zone, i.e. | c_i|>The information of the distortion of the TH,

can be determined based on the following equations (7) and (8):

wherein,

representing a quantization offset;

representing the coefficients at position (x, y) in an image block of size S;

the reference coefficient at a position (x, y) in the image block with the size S is shown, which is not limited in the embodiment of the present application;

representing distribution parameters after fitting the residual distribution to a Laplace distribution, i.e.

It should be noted that the above description of the method for predicting the bitrate and the distortion information of the video frame based on the reference model is only an exemplary description of one possible implementation manner, and the embodiment of the present disclosure does not limit the specific structure of the reference model and the specific prediction method of the bitrate and the distortion.

In step 203, the computer device adjusts the initial quantization parameter corresponding to each video frame based on the matching relationship between the predicted code rate and the predicted distortion information and the second reference condition, so as to obtain the quantization parameter corresponding to each video frame.

The second reference condition may be set by a developer, for example, the second reference condition includes numerical limitation information on the prediction code rate and the prediction distortion information, which is not limited in the embodiments of the present disclosure.

In one possible implementation, in response to the sum of the prediction code rates being greater than a first code rate threshold, the computer device may increase the value of the initial quantization parameter corresponding to each video frame, and determine the prediction code rate and the prediction distortion information corresponding to each video frame based on the adjusted initial quantization parameter until the sum of the prediction code rates corresponding to each video frame is less than the first code rate threshold; in response to that the sum of the predicted code rates is smaller than the first code rate threshold and that the fluctuation amplitude of each piece of predicted distortion information is smaller than a fluctuation threshold, the computer device may determine the initial quantization parameter currently corresponding to each video frame as the quantization parameter; in response to that the sum of the prediction code rates is smaller than the first code rate threshold and the fluctuation amplitude of each piece of prediction distortion information is larger than the fluctuation threshold, the computer device may decrease the initial quantization parameter of the video frame corresponding to the largest piece of prediction distortion information, increase the initial quantization parameter of the video frame corresponding to the smallest piece of prediction distortion information, and obtain the quantization parameter corresponding to each video frame. The first code rate threshold and the fluctuation threshold may be set by a developer, which is not limited in the embodiment of the disclosure.

Fig. 3 is a schematic diagram illustrating a method for adjusting quantization parameter at video frame level according to an exemplary embodiment, and the quantization parameter adjustment process is described with reference to fig. 3, where first, a computer device obtains a plurality of video frames, and for a video frame i, based on an initial quantization parameter QP_iAfter the residual block data of the video frame i is predicted to be subjected to DCT or DST, the corresponding residual coefficient is predicted, that is, step 301 is executed, the residual coefficient corresponding to the residual block data of the video frame i is input into a reference model 302, the prediction code rate and the prediction distortion information corresponding to each image block are predicted, the prediction code rate and the prediction distortion information of the image block level are accumulated, and the prediction code rate of the frame level is obtained

And predicting distortion information

Then, the computer device obtains the sum of the code rates corresponding to the plurality of video frames

And sum of distortion information

Executing step 303, judging whether the sum of the code rates of the plurality of video frames is greater than a first code rate threshold, if so, executing step 304, adding one to the initial quantization parameter of each video frame, inputting the adjusted initial quantization parameter into the reference model 302, and executing the subsequent steps again; if not, step 305 is executed to obtain the fluctuation range of each predicted distortion information, i.e. the predicted distortion information with the largest valueInformation processing device

Prediction distortion information with minimum value

And judging whether the fluctuation amplitude is smaller than a fluctuation threshold value. If not, executing step 306, subtracting one from the initial quantization parameter of the video frame corresponding to the maximum prediction distortion information, adding one to the initial quantization parameter of the video frame corresponding to the minimum prediction distortion information, inputting the updated initial quantization parameter into the reference model 302, and executing the subsequent steps again; and if not, determining the initial quantization parameter corresponding to each current video frame as the quantization parameter corresponding to each video frame.

It should be noted that, the steps 201 to 203 are steps in which the computer device obtains a plurality of video frames to be encoded and a quantization parameter corresponding to each video frame. In the embodiment of the disclosure, an initial quantization parameter is first specified, code rate and distortion information corresponding to each video frame after encoding are predicted by a reference model based on the initial quantization parameter, and the initial quantization parameter of each video frame is adjusted according to the predicted code rate and distortion information to adjust the code rate corresponding to each video frame, so that the total code rate corresponding to the video can meet the requirement.

In step 204, for any video frame in the plurality of video frames, the computer device encodes the any video frame based on the quantization parameter corresponding to the any video frame to obtain the actual bitrate and the actual distortion information corresponding to the any video frame.

In this disclosure, the computer device may encode each image block in any video frame based on the quantization parameter corresponding to the video frame to obtain an intermediate code rate and intermediate distortion information corresponding to each image block. In a possible implementation manner, for a first image block in any one of the video frames, the computer device may directly encode the first image block based on a quantization parameter corresponding to the any one of the video frames to obtain an intermediate bitrate and intermediate distortion information corresponding to the first image block; for any image block in the any video frame except the first image block, the computer device may obtain an intermediate code rate and intermediate distortion information corresponding to a previous image block, adjust a quantization parameter currently corresponding to the any video frame based on the intermediate code rate and intermediate distortion information corresponding to the previous image block, and encode the any image block based on the adjusted quantization parameter.

Fig. 4 is a schematic diagram of a method for adjusting quantization parameters at an image block level according to an exemplary embodiment, and the process of adjusting quantization parameters at an image block level is described with reference to fig. 4, in a possible implementation manner, a computer device first determines whether an intermediate code rate corresponding to a previous image block is greater than a second code rate threshold, that is, performs step 401. In response to that the intermediate code rate corresponding to the previous image block is greater than the second code rate threshold, the computer device executes step 402, that is, adds one to the numerical value of the quantization parameter to obtain an adjusted quantization parameter, and continues to encode any image block based on the adjusted quantization parameter. In response to the intermediate code rate corresponding to the previous image block being less than the second code rate threshold, the computer device continues to adjust the quantization parameter based on a magnitude relationship between the intermediate distortion information and the intermediate distortion threshold. That is, the computer device executes step 403 to determine whether the intermediate distortion information is greater than the intermediate distortion threshold, and if not, encodes any image block based on the current quantization parameter; if yes, a first error value between the intermediate code rate and the second code rate threshold is obtained, and step 404 is executed to determine whether the first error value is greater than a first error threshold. In response to that the first error value is greater than the first error threshold, subtracting one from the value of the quantization parameter, that is, performing step 405 to obtain the adjusted quantization parameter, and encoding the image block based on the adjusted quantization parameter; otherwise, the current quantization parameter does not need to be adjusted, and any image block is directly encoded based on the current quantization parameter. The second code rate threshold and the first error threshold may be set by a developer, which is not limited in this disclosure. It should be noted that, the specific method for encoding the image block in the embodiment of the present disclosure is not limited. In the embodiment of the disclosure, when any video frame is encoded, based on the actual code rate and the actual distortion information corresponding to the encoded image block, the quantization parameter applied to the subsequent image block encoding is adjusted to level the code rate distribution among the image blocks, thereby realizing the control of the video frame code rate, avoiding the situation of insufficient code rate or overlarge code rate in the encoding process, and ensuring that the total code rate corresponding to the video meets the requirement.

In the embodiment of the present disclosure, after the computer device completes encoding each image block in one video frame and obtains the intermediate code rate and the intermediate distortion information corresponding to each image block, the actual code rate and the actual distortion information corresponding to any video frame may be determined based on the intermediate code rate and the intermediate distortion information corresponding to each image block. For example, the intermediate code rates are accumulated to obtain the actual code rate corresponding to the video frame, and the distortion information is accumulated to obtain the actual distortion information corresponding to the video frame.

In step 205, the computer device adjusts a quantization parameter corresponding to a next video frame based on the actual bitrate corresponding to the any video frame and a matching relationship between the actual distortion information and a first reference condition, and encodes the next video frame based on the adjusted quantization parameter.

The first reference condition may be set by a developer, for example, the first reference condition includes numerical limitation information on an actual code rate and actual distortion information, and the like, which is not limited in the embodiments of the present disclosure.

In a possible implementation manner, the computer device first determines a numerical relationship between an actual code rate corresponding to the any video frame and the third code rate threshold. And in response to the actual code rate corresponding to any video frame being greater than a third code rate threshold, adding one to the quantization parameter corresponding to the next video frame. In response to that the actual code rate corresponding to the any video frame is smaller than the third code rate threshold, the computer device adjusts the quantization parameter corresponding to the next video frame based on a size relationship between the actual distortion information corresponding to the any video frame and the actual distortion threshold. In a possible implementation manner, the computer device may determine a numerical relationship between actual distortion information of any one of the video frames and an actual distortion threshold, and obtain a second error value between an actual code rate of the any one of the video frames and a third code rate threshold in response to that the actual distortion information corresponding to the any one of the video frames is greater than the actual distortion threshold; in response to the second error value being greater than a second error threshold, one is decreased for the quantization parameter corresponding to the next video frame. It should be noted that the above process of adjusting the quantization parameter at the video frame level is the same as the process of adjusting the quantization parameter at the image block level shown in fig. 4, and details are not repeated here.

It should be noted that, in this embodiment of the present disclosure, if the video frame to be encoded is the first frame in the video, or the actual code rate and the actual distortion information of the previous video frame are not obtained, the code rate and the distortion of the video frame to be encoded may be predicted based on the specified initial quantization parameter, and then the initial quantization parameter is adjusted based on the predicted data to obtain the quantization parameter corresponding to the video frame to be encoded, and the quantization parameter is applied to encode the video frame to be encoded, where the specific process is as in steps 201 to 204 described above. If the video to be encoded is not the first frame in the video and the actual bitrate and the actual distortion information of the previous video frame can be obtained, the quantization parameter of the video frame to be encoded can be adjusted based on the actual bitrate and the actual distortion information of the previous video frame, and the specific process is as described in step 205 above.

Fig. 5 is a flowchart illustrating a rate control method according to an exemplary embodiment, and the video data processing procedure is described with reference to fig. 5. For any video frame, firstly, the computer device performs a video frame preprocessing step 501 to obtain residual coefficients of image block levels in the video frame, inputs the obtained residual coefficients and initial quantization parameters into a reference model 502, performs code rate prediction and distortion information prediction by the reference quantization model, and performs a code rate allocation step 503 of the video frame level based on the predicted code rate and distortion information, that is, the process of the step 203 for adjusting the initial quantization parameters, to obtain quantization parameters of the video frame level. Then, the computer device executes step 504 of encoding one or a row of image blocks in the video frame to obtain the actual code rate and the actual distortion information of the image block level, and then executes step 505 of adjusting the quantization parameter of the image block level, that is, the process of implementing step 204 described above. Finally, the computer device, in response to that the encoding of any video frame is completed and that any video frame is not the last frame, performs step 503 based on the actual bitrate and the actual distortion information corresponding to any video frame, and controls the bitrate of the next video frame. According to the technical scheme provided by the embodiment of the disclosure, the quantization parameter of the uncoded video frame is adjusted based on the code rate and distortion information of the coded video frame, so that the compression efficiency of the video coder is improved, and the code rate of the video is reduced under the condition of ensuring the video quality; and the quantization parameters of the video frame level and the quantization parameters of the video block level are adjusted, namely, the code rate of each video frame is controlled more accurately through rough adjustment and precise adjustment, the requirement of the change bandwidth is met, the buffer (buffer) is prevented from overflowing up and down, a smoother video is obtained, and the video quality is ensured not to change suddenly in the playing process of the video.

Fig. 6 is a block diagram illustrating a video data processing apparatus according to an example embodiment. Referring to fig. 6, the apparatus includes a first obtaining module 601, a second obtaining module 602, and an adjusting module 603.

A first obtaining module 601, configured to obtain a plurality of video frames to be encoded and a quantization parameter corresponding to each video frame;

a second obtaining module 602, configured to, for any video frame in the multiple video frames, encode the any video frame based on a quantization parameter corresponding to the any video frame, so as to obtain an actual bitrate and actual distortion information corresponding to the any video frame;

an adjusting module 603 configured to adjust a quantization parameter corresponding to a next video frame based on the actual bitrate corresponding to the any video frame and a matching relationship between the actual distortion information and a first reference condition, and encode the next video frame based on the adjusted quantization parameter.

In one possible implementation, the first obtaining module 601 includes:

In one possible implementation, the first determining sub-module includes:

In one possible implementation, the prediction unit is configured to:

In one possible implementation, the adjustment submodule is configured to:

In one possible implementation, the second obtaining module 602 includes:

In one possible implementation, the encoding submodule includes:

In one possible implementation, the second encoding unit includes:

In one possible implementation, the second adjusting subunit is configured to:

In one possible implementation, the adjusting module 603 includes:

In one possible implementation, the apparatus is applied to a video encoder.

According to the device provided by the embodiment of the disclosure, after one video frame is encoded, the quantization parameter applied by the subsequent video frame encoding is adjusted based on the actual code rate and the actual distortion information of the video frame, so that the code rate of the subsequent video frame after encoding is controlled, the total code rate of the video frame after encoding can be controlled in a reasonable range, and the actual distortion information of the video frame is considered in the quantization parameter adjusting process, so that the total code rate of the video can meet the code rate requirement under the condition of ensuring the quality of the encoded video.

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

FIG. 7 is a block diagram illustrating a computer device in accordance with an example embodiment. The computer device 700 may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 701 and one or more memories 702, where the memory 702 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 701 to implement the video data processing method provided by the above-mentioned method embodiments. Certainly, the computer device may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the computer device may further include other components for implementing the functions of the device, which is not described herein again.

In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as a memory comprising instructions, executable by a processor of a computer device to perform the above-described method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, there is also provided a computer program product comprising executable instructions which, when executed by a processor of a computer device, enable the computer device to perform any of the video data processing methods as described above.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of processing video data, comprising:

and adjusting the quantization parameter corresponding to the next video frame based on the actual code rate corresponding to any video frame and the matching relation between the actual distortion information and a first reference condition, and encoding the next video frame based on the adjusted quantization parameter.

2. The method according to claim 1, wherein said obtaining a plurality of video frames to be encoded and a quantization parameter corresponding to each video frame comprises:

3. The method of claim 2, wherein the determining the prediction code rate and the prediction distortion information corresponding to each video frame based on the initial quantization parameter and the pixel information corresponding to each video frame comprises:

based on the correlation among the pixel points in each video frame, carrying out intra-frame prediction and inter-frame prediction on each video frame to obtain predicted image data corresponding to each video frame;

and determining a prediction code rate and prediction distortion information corresponding to each video frame based on the prediction residual data corresponding to each video frame and the initial quantization parameter.

4. The method of claim 3, wherein the determining the prediction code rate and the prediction distortion information corresponding to each video frame based on the prediction residual data corresponding to each video frame and the initial quantization parameter comprises:

determining a plurality of residual coefficients corresponding to each video frame based on the prediction residual data corresponding to each video frame;

5. The method of claim 1, wherein the encoding the any video frame based on the quantization parameter corresponding to the any video frame to obtain an actual bitrate and actual distortion information corresponding to the any video frame comprises:

coding each image block in any video frame based on the quantization parameter corresponding to any video frame to obtain an intermediate code rate and intermediate distortion information corresponding to each image block;

6. The method according to claim 5, wherein the encoding each image block in any video frame based on the quantization parameter corresponding to any video frame to obtain an intermediate code rate and intermediate distortion information corresponding to each image block comprises:

7. The method of claim 1, wherein the adjusting quantization parameters corresponding to a next video frame based on the actual bitrate and the matching relationship between the actual distortion information and a first reference condition corresponding to the any video frame comprises:

and in response to that the actual code rate corresponding to any video frame is smaller than the third code rate threshold, adjusting the quantization parameter corresponding to the next video frame based on the size relationship between the actual distortion information corresponding to any video frame and the actual distortion threshold.

8. A video data processing apparatus, comprising:

a second obtaining module, configured to encode, for any video frame in the plurality of video frames, the any video frame based on a quantization parameter corresponding to the any video frame to obtain an actual bitrate and actual distortion information corresponding to the any video frame;

an adjusting module configured to adjust a quantization parameter corresponding to a next video frame based on the actual bitrate corresponding to the any video frame and a matching relationship between the actual distortion information and a first reference condition, and encode the next video frame based on the adjusted quantization parameter.

9. A computer device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the video data processing method of any of claims 1 to 7.

10. A computer-readable storage medium in which instructions, when executed by a processor of a computer device, enable the computer device to perform the video data processing method of any one of claims 1 to 7.