WO2017140230A1 - Method and device for adjusting target code rate - Google Patents

Abstract

A method for adjusting a target code rate. The method for adjusting a target code rate comprises: dividing an image frame to be encoded into a plurality of first sub-images, classifying the plurality of first sub-images according to a motion vector value of each of the first sub-images, and calculating a peak signal-to-noise ratio (PSNR) value of each class of first sub-images; calculating an average value of the PSNR value of each class of first sub-images, calculating the standard deviation of the PSNR value of each class of first sub-images, and correcting the ratio of the average value to the standard deviation by using a pre-set correction parameter, and then taking the ratio as the fluctuation strength of the encoding; and adjusting a target code rate for encoding according to a magnitude relationship between the fluctuation strength and a first pre-set threshold value, wherein when the fluctuation strength is greater than the first pre-set threshold value, the target code rate for encoding is increased, and when the fluctuation strength is less than the first pre-set threshold value, the target code rate for encoding is reduced. The technical solution can improve the accuracy of selecting a target code rate.

Description

Target rate adjustment method and device Technical field

This document relates to, but is not limited to, the field of video coding technology, and relates to a target rate adjustment method and apparatus.

Background technique

In the variable rate control of video coding, the selection of the target bit rate is an important link, which is directly related to the visual quality of the picture. In the related art, the target bit rate of the encoded code is usually controlled based on the network bandwidth to obtain a balance between visual quality and bandwidth usage. Specifically, the code rate controller sets the capacity feedback value of the hypothetical reference decoding buffer based on the network bandwidth, and is used for The encoded target code rate is calculated and assigned to maintain a smooth video output stream as much as possible, and to provide a minimum distortion video image decoding visual quality at a minimum bit rate. However, since the related art selects the target code rate based on the capacity of the imaginary reference decoding buffer, there is a problem that the selected target code rate is not accurate enough.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

The embodiment of the invention provides a method and a device for adjusting a target bit rate, which can improve the accuracy of the selected target bit rate.

An embodiment of the present invention provides a target rate adjustment method, where the target rate adjustment method includes:

Dividing the image frame to be encoded into a plurality of first sub-images, and classifying the plurality of first sub-images according to motion vector values of each of the first sub-images, and calculating peak values of each of the first sub-images Signal to noise ratio PSNR value;

Calculating an average value μ of PSNR values of each type of first sub-image, and calculating a standard deviation σ of PSNR values of each type of first sub-image, and correcting the ratio of σ to μ as a coded fluctuation intensity by using a preset correction parameter;

And adjusting a target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, wherein the adjusting the target code rate according to the magnitude relationship between the fluctuation strength and the first preset threshold include:

When the fluctuation strength is greater than the first preset threshold, increasing the encoded target code rate;

When the fluctuation strength is less than the first predetermined threshold, the encoded target code rate is reduced.

Optionally, before the step of adjusting the target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, the method further includes:

Recording the currently calculated fluctuation intensity, and calculating an average value of the recorded fluctuation strength when the recorded fluctuation intensity reaches the second preset threshold;

The encoded wave strength is updated to the calculated average.

Optionally, the image frame to be encoded is divided into a plurality of first sub-images, and the plurality of first sub-images are classified according to motion vector values of each of the first sub-images, and each class is calculated. The steps of the PSNR value of the first sub-image include:

Dividing the image frame to be encoded into a plurality of first sub-images, and calculating a motion vector value of each of the first sub-images of the image frame to be encoded relative to the adjacent image frames, according to each of the first sub- Sorting the plurality of first sub-images by a vector value interval in which a motion vector value of the image is located;

Decomposing the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images in the same division manner as the image frame to be encoded, and calculating each of the first sub-images and each of the first a mean square error between the second sub-images corresponding to a sub-image;

A PSNR value for each type of first sub-image is calculated based on each of the mean square errors.

Optionally, after the step of adjusting the target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, the method further includes:

The adjusted target code rate is corrected based on the current network bandwidth to be encoded using the corrected target code rate.

Optionally, the step of modifying the adjusted target bit rate based on the current network bandwidth includes:

Determining whether the current network bandwidth is greater than the adjusted target code rate;

When the current network bandwidth is less than the adjusted target code rate, the network bandwidth is used as the corrected target code rate.

The embodiment of the present invention provides a target rate adjustment apparatus, where the target rate adjustment apparatus includes:

a first calculating module, configured to divide the image frame to be encoded into a plurality of first sub-images, and classify the plurality of first sub-images according to motion vector values of each of the first sub-images, and calculate each Peak signal to noise ratio PSNR value of the first sub-image of the class;

a second calculating module, configured to calculate an average value μ of PSNR values of each type of first sub-image, and calculate a standard deviation σ of PSNR values of each type of first sub-image, and correct the ratio of σ to μ by using a preset correction parameter After the wave strength as the code;

The adjusting module is configured to adjust the target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, wherein the target code rate of the encoding is adjusted according to the magnitude relationship between the fluctuation strength and the first preset threshold include:

When the fluctuation strength is greater than the first preset threshold, increasing the encoded target code rate;

When the fluctuation strength is less than the first predetermined threshold, the encoded target code rate is reduced.

Optionally, the target rate adjustment device further includes:

a recording module configured to record the currently calculated fluctuation intensity, and calculate an average value of the recorded fluctuation strength when the recorded fluctuation intensity reaches a second preset threshold;

An update module is arranged to update the encoded wave strength to the calculated average.

Optionally, the first computing module includes:

a classifying unit configured to divide the image frame to be encoded into a plurality of first sub-images, and calculate a motion vector value of each of the first sub-images relative to the adjacent image frames of the image frame to be encoded, according to each Sorting the plurality of first sub-images by a vector value interval in which the motion vector value of the first sub-image is located;

a calculating unit, configured to divide the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images in the same division manner as the image frame to be encoded, and calculate each of the first sub-images and a mean square error between the second sub-images corresponding to each of the first sub-images; and calculating a PSNR value for each of the first sub-images based on each of the mean square errors.

Optionally, the target rate adjustment apparatus further includes a correction module configured to correct the adjusted target code rate based on the current network bandwidth to perform encoding using the corrected target code rate.

Optionally, the modifying module corrects the adjusted target code rate based on the current network bandwidth by determining whether the current network bandwidth is greater than the adjusted target code rate; and in the current network. When the bandwidth is less than the adjusted target code rate, the network bandwidth is used as the corrected target code rate.

In the embodiment of the present invention, the image frame to be encoded is divided into a plurality of first sub-images, and the plurality of the first sub-images are classified according to the motion vector values of the first sub-images, and the first type is calculated. The peak signal-to-noise ratio PSNR value of the sub-image, and then the ratio of the standard deviation σ and the average value μ of the PSNR values of the first sub-images are corrected by the preset correction parameters as the encoded fluctuation intensity, and the obtained fluctuation intensity includes Reconstruction distortion of image frames and features of motion compensation. The above technical solution, when the fluctuation strength is increased in the encoding process, based on the time domain correlation of the adjacent image frames, the next image frame to be encoded of the image frame to be encoded needs more bits for description (ie, encoding). Therefore, compared with the related art, the target code rate is selected by the imaginary value, and the technical solution of the embodiment of the present invention can accurately reflect the bit required for the coding by adjusting the target code rate of the coded by the fluctuation intensity in real time, thereby improving the selected target. The accuracy of the bit rate. The embodiment of the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and the computer executable instructions are implemented to implement a target rate adjustment method.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

DRAWINGS

1 is a schematic flowchart of a target bit rate adjustment method according to Embodiment 1 of the present invention;

2 is a schematic diagram of motion compensation in a first embodiment of a target code rate adjustment method according to Embodiment 1 of the present invention;

3 is a real measurement diagram of target rate adjustment in the first embodiment of the code rate adjustment method according to the first embodiment of the present invention;

4 is a schematic diagram showing a refinement flow of calculating a PSNR value of each type of first sub-image in FIG. 1;

5 is a motion vector distribution diagram of a first sub-image of a target bit rate adjustment method according to Embodiment 3 of the present invention;

6 is a schematic diagram of functional blocks of a target bit rate adjustment apparatus according to Embodiment 5 of the present invention;

FIG. 7 is a schematic diagram of a refinement function module of the first computing module in FIG. 6.

detailed description

It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.

Embodiment 1

An embodiment of the present invention provides a target rate adjustment method. Referring to FIG. 1, the target rate adjustment method includes:

Step S10, dividing the image frame to be encoded into a plurality of first sub-images, and classifying the plurality of first sub-images according to motion vector values of each of the first sub-images, and calculating the first sub-category of each Peak signal to noise ratio PSNR value of the image;

The target rate adjustment method proposed in this embodiment can be applied to video coding control of streaming media, for example, when encoding, by motion compensation vector calculation, and combining the distortion statistics of the reconstructed image frame to perform coding of the target code rate. Select to accurately select the appropriate target bit rate to reduce the occupation of network bandwidth while ensuring video quality.

Since one continuous video is composed of a series of image frames, in order to facilitate calculation, in the present embodiment, the image frame to be encoded is divided into a plurality of first sub-images in a square shape. It can be understood by those skilled in the art that in other embodiments, the shape of the first sub-image may be selected according to actual needs. For example, the image frame may be divided into a plurality of first sub-images that are rectangular.

In this embodiment, the number of the first sub-images divided depends on the size of the first sub-image, wherein the pixel width of the image frame is W, the pixel height is H, and the size of the first sub-image is n. *n is an example, and the number of the first sub-images is divided into N=WB×HB, wherein

Round up the operation. Optionally, in order to increase the speed of the processing, the size of the first sub-image may be set to 64*64.

After dividing the image frame into a plurality of first sub-images, calculating motion vector values of each of the first sub-images, and pairing the plurality of the first ones according to motion vector values of each of the first sub-images One son The image is classified, that is, a plurality of the first sub-images are classified according to a vector value interval in which the motion vector value of each of the first sub-images is located. Among them, the motion vector value can directly reflect the fluctuation strength of the image.

It should be noted that in video coding, the image content of the moving image adjacent to the image frame has time domain correlation. In this embodiment, motion compensation is performed for each of the first sub-images, a matching block of the first sub-image is searched for in an adjacent image frame of the image frame, and a motion vector of the matching block of the first sub-image is recorded. MV _i,j =(Δx,Δy),

Is a motion vector value, where i is the width of the image, j is the height of the image, Δx represents the displacement of the first sub-image in the horizontal axis (x-axis) direction, and Δy represents the first sub-image on the vertical axis ( The displacement in the y-axis) direction. For example, referring to FIG. 2, a first sub-image is selected in the image frame at time t, a minimum distortion search is performed in the adjacent image frame at time t-1, a matching block with minimum distortion is obtained, and the first sub-child is calculated. The vector distance of the matching block of the image and the minimum distortion, as shown in Fig. 2, MV = (Δx, Δy), thereby calculating the motion vector value of the selected first sub-image

Moreover, those skilled in the art can understand that the adjacent image frame of the currently selected image frame refers to the previous image frame in which the image frame is selected on the frame sequence.

After the classification of the first sub-image is completed, a peak signal to noise ratio PSNR (Peak Signal to Noise Ratio) value of each type of first sub-image is calculated. Among them, PSNR is the most common and widely used objective standard for evaluating images.

Step S20, calculating an average value μ of PSNR values of each type of first sub-image, and calculating a standard deviation σ of PSNR values of each type of first sub-image, and correcting the ratio of σ to μ by using a preset correction parameter as a coded Fluctuation intensity

In this embodiment, the average value of the PSNR values of each type of first sub-image

Standard deviation of PSNR values for each type of first sub-image

Where k is an integer, each vector value interval corresponds to a k value, and P _k is a PSNR value of each type of first sub-image.

After calculating the σ value and the μ value of each type of sub-image, the ratio of σ to μ is corrected using the preset correction parameter φ as the encoded fluctuation intensity V, that is, the fluctuation intensity.

The φ value is an empirical parameter for normalizing the ratio of σ to μ to increase the processing speed. For example, the φ value is set to 100 in this embodiment.

In addition, in order to increase the processing speed, a maximum k value k _max may also be set, and a first sub-image with a motion vector value greater than k _max is divided into k _max intervals, where k _max is an empirical parameter, for example, this embodiment will k _{max is} set to 9.

In step S30, the target code rate of the encoding is adjusted according to the magnitude relationship between the fluctuation strength and the first preset threshold. The target code rate of the encoding is adjusted according to the magnitude relationship between the fluctuation strength and the first preset threshold.

When the fluctuation strength is greater than the first preset threshold, increasing the encoded target code rate;

When the fluctuation strength is less than the first predetermined threshold, the encoded target code rate is reduced.

It will be understood by those skilled in the art that in video coding, an image frame having a large fluctuation intensity requires more bits to be described than an image frame having a small fluctuation intensity. Therefore, in this embodiment, after the calculated fluctuation strength V is calculated, it is determined whether the fluctuation strength V is greater than a first preset threshold, and if the fluctuation strength V is greater than the first preset threshold, the encoded target code is increased. Rate; otherwise, if the fluctuation strength V is less than or equal to the first predetermined threshold, the encoded target code rate is reduced. The first preset threshold may be set according to actual needs. For example, in this embodiment, the first preset threshold is set to 5. In this embodiment, the step value TBR _{C of the} target rate adjustment is preset. As a basis for each target rate adjustment.

For example, referring to FIG. 3, a measured view of the target code rate adjustment of the embodiment is provided. As shown in FIG. 3, in the encoding process, the target code rate is adjusted according to the change of the fluctuation intensity in real time. It is possible that the low bit occupancy occupies the visual decoding quality of the video image with minimal distortion.

Further, when the fluctuation intensity V is equal to the preset threshold, the encoded target code rate is not adjusted.

In other embodiments, a first preset threshold interval may be further set, when the fluctuation strength V is greater than a maximum value in the first preset threshold interval, increasing a target code rate of the encoding; when the fluctuation strength is When the V is located in the first preset threshold interval, the encoded target code rate is not adjusted; when the fluctuation strength V is smaller than the minimum value in the first preset threshold interval, the encoded target code rate is reduced. For example, the first preset interval is set to [4, 5].

The target rate adjustment method proposed in this embodiment divides an image frame to be encoded into a plurality of first a sub-image, and classifying the plurality of first sub-images according to motion vector values of each of the first sub-images, calculating a peak signal-to-noise ratio PSNR value of each type of first sub-image, and then first The ratio of the standard deviation σ of the PSNR value of the sub-image and the average value μ is corrected by the preset correction parameter as the encoded fluctuation intensity, and the obtained fluctuation intensity includes the reconstruction distortion of the image frame and the characteristics of the motion compensation. When the fluctuation strength increases during the encoding process, based on the time domain correlation of the adjacent image frames, the next image frame to be encoded of the image frame to be encoded requires more bits for description (ie, encoding), therefore, Compared with the related art, the target code rate is selected by the imaginary value. The technical solution of the embodiment of the present invention can accurately reflect the bit required for the coding by adjusting the target code rate of the coded in real time by the fluctuation strength, and improve the selected target bit rate. Accuracy.

Embodiment 2

Based on the first embodiment, in the embodiment, before the step S30, the target rate adjustment method further includes:

Recording the currently calculated fluctuation intensity, and calculating an average value of the recorded fluctuation strength when the recorded fluctuation intensity reaches the second preset threshold;

The encoded wave strength is updated to the calculated average.

Since the image content of the moving image adjacent to the image frame has time domain correlation, the image content of the adjacent consecutive multiple image frames tends to be substantially the same, and correspondingly, the variation of the fluctuation intensity of the encoding in a short time is often negligible. In practical applications, although the fluctuation strength of the encoding is calculated once per frame, the target rate is adjusted once in a long time, and there is a waste of processing resources. Therefore, in this embodiment, a window having a size of τ (the aforementioned second preset threshold, specifically, can be set as needed) is set, and the average fluctuation intensity of the image frame in the window is counted every τ frame to determine whether to perform The adjustment of the target bit rate. Where τ is a positive number greater than 0, for example, the value range can be set to [1, 30].

In this embodiment, after completing the encoding of the image frame to be encoded and calculating the fluctuating intensity of the encoding, the fluctuation intensity of the current calculation is recorded, and when the calculated fluctuation intensity of the τ is recorded, the image frame within the window is counted. Average fluctuation intensity

Update the encoded wave strength to the calculated average wave strength

For the adjustment of the target code rate of the coding, reference may be made to the foregoing embodiment, and details are not described herein again. Where t is an integer multiple of τ, and t is used to indicate the position of the corresponding image frame in the sequence of frames.

In other embodiments, when the currently calculated fluctuation strength is recorded, the currently calculated fluctuation intensity may also be smoothed based on the last calculated fluctuation intensity to eliminate the influence of noise during the encoding process. In this embodiment, the current The calculated fluctuation intensity is recorded as: a·V _t-1 +b·V _t ;

Where a and b are empirical parameters (for example, in this embodiment, a takes a value of 0.1 and b takes a value of 0.9). V _t-1 represents the last calculated fluctuation intensity, and V _t represents the currently calculated fluctuation intensity.

In this embodiment, it is determined whether the target code rate needs to be adjusted and how to adjust by a certain interval, and the processing resource consumption is reduced on the basis of ensuring the accuracy of selecting the target code rate.

Embodiment 3

Optionally, based on the first embodiment, referring to FIG. 4, in the embodiment, the foregoing step S10 includes:

Step S101, dividing an image frame to be encoded into a plurality of first sub-images, and calculating a motion vector value of each of the first sub-images of the image frame to be encoded relative to adjacent image frames, according to each of the Sorting the plurality of first sub-images by a vector value interval in which a motion vector value of a sub-image is located;

A continuous video is composed of a series of image frames. For the convenience of calculation, in the embodiment, the image frame to be encoded is divided into a plurality of first sub-images in a square shape. It can be understood by those skilled in the art that in other embodiments, the shape of the first sub-image may be selected according to actual needs. For example, the image frame may be divided into a plurality of first sub-images that are rectangular. In this embodiment, the number of the first sub-images divided depends on the size of the first sub-image, wherein the pixel width of the image frame is W, the pixel height is H, and the size of the first sub-image is n. *n is an example, and the number of the first sub-images is divided into N=WB×HB, wherein

Round up the operation. Optionally, in order to increase the speed of the processing, the size of the first sub-image may be set to 64*64.

After dividing the image frame into a plurality of first sub-images, calculating motion vector values of each of the first sub-images, and pairing the plurality of the first ones according to motion vector values of each of the first sub-images A sub-image is classified, that is, a plurality of the first sub-images are classified according to a vector value interval in which the motion vector value of each of the first sub-images is located. Among them, the motion vector value can directly reflect the fluctuation strength of the image.

It should be noted that in video coding, the image content of the moving image adjacent to the image frame has a time domain correlation. In this embodiment, motion compensation is performed for each of the first sub-images, a matching block of the first sub-image is searched for in an adjacent image frame of the image frame, and a motion vector of the matching block of the first sub-image is recorded. MV _i,j =(Δx,Δy),

Is a motion vector value, where Δx represents the displacement of the first sub-image in the x-axis direction, and Δy represents the displacement of the first sub-image in the y-axis direction. For example, referring to FIG. 2, a first sub-image is selected in the image frame at time t, and a minimum distortion search is performed in the adjacent image frame at time t-1 to obtain a minimum distortion matching block, and the first first is calculated. The vector distance of the sub-image and the minimum distortion matching block, as shown in Fig. 2, MV = (Δx, Δy), thereby calculating the motion vector value of the selected first sub-image

The calculated motion vector distribution of each of the first sub-images of the image frame is as shown in FIG.

Step S102, dividing the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images according to the same division manner as the image frame to be encoded, and calculating each of the first sub-images and each a mean square error between the second sub-images corresponding to the first sub-image;

It will be understood by those skilled in the art that in inter prediction coding, the next frame of coding is encoded based on the reconstructed frame of the current frame. In this embodiment, the reconstructed image frame of the image frame to be encoded is divided into a plurality of second sub-images in the same division manner as the image frame to be encoded, and each of the first sub-images is calculated. And a mean square error between the second sub-images corresponding to each of the first sub-images, wherein a mean square error between the first sub-image and its corresponding second sub-image

Represents the standard deviation of the first sub-image,

Indicates the standard deviation of the second sub-image.

Step S103, calculating a PSNR value of each class first sub-image based on each of the mean square errors.

In this embodiment, the average distortion D _{k of the} first sub-image of each class is first counted as ∑D _i,j |k<||MV _i,j ||<k+1, where

For the motion vector size, k is an integer, and each of the aforementioned vector value intervals corresponds to a k value.

Then calculating the PSN R value of the first sub-image of each class based on the average distortion of the first sub-image of each class, and the PSNR value of the first sub-image of each class

Where bpp represents the number of bits when the pixel point gray value is expressed in binary, as in the 0 to 255 gray level representation, the value of bpp is 8.

Embodiment 4

Optionally, based on any of the foregoing embodiments, in the embodiment, after the step S30, the target rate adjustment method further includes:

The adjusted target code rate is corrected based on the current network bandwidth to be encoded using the corrected target code rate.

It should be noted that the network bandwidth in this embodiment refers to the network bandwidth available for encoding video transmission, and those skilled in the art may understand that the network bandwidth is dynamically changed, and the network bandwidth is also local resources. The problem of preemption, therefore, in order to ensure the smoothness of the transmission of the encoded video, in this embodiment, the currently available network bandwidth is obtained, and the adjusted target bit rate is corrected based on the obtained network bandwidth, and the method is adopted. The corrected target bit rate is encoded.

Optionally, the correcting the adjusted target bit rate based on the current network bandwidth includes:

Determining whether the current network bandwidth is greater than the adjusted target code rate;

When the current network bandwidth is less than the adjusted target code rate, the network bandwidth is used as the corrected target code rate.

It is easy to understand that when the network bandwidth is smaller than the adjusted target code rate, if the adjusted target bit rate is used for encoding, the encoded video data cannot be smoothly transmitted to the target terminal, resulting in playback. Problems such as Caton affect the user's visual experience. Therefore, in this embodiment, when the current network bandwidth is less than the adjusted target code rate, the network bandwidth is used as the corrected target code rate to ensure that the encoded video data can be smoothly transmitted to Target terminal.

Embodiment 5

An embodiment of the present invention provides a target rate adjustment apparatus. Referring to FIG. 6, the target rate adjustment apparatus includes:

The first calculating module 10 is configured to divide the image frame to be encoded into a plurality of first sub-images, and classify the plurality of first sub-images according to motion vector values of each of the first sub-images, and calculate Peak signal to noise ratio PSNR value of the first sub-image of each class;

The target rate adjustment apparatus proposed in this embodiment may be applied to video coding control of a streaming media, for example, at the time of encoding, by performing motion compensation vector calculation, and combining the distortion statistics of the reconstructed image frame to perform coding of the target code rate. Select to accurately select the appropriate target bit rate to ensure video quality The amount of time reduces the occupation of network bandwidth.

Since a continuous video is composed of a series of image frames, in order to facilitate calculation, in the embodiment, the first calculation module 10 divides the image frame to be encoded into a plurality of first sub-images in a square shape. It can be understood by those skilled in the art that in other embodiments, the shape of the first sub-image may be selected according to actual needs. For example, the first calculation module 10 may divide the image frame into a plurality of rectangles. Sub image.

In this embodiment, the number of the first sub-images obtained by the first calculation module 10 depends on the size of the first sub-image, and the pixel width of the image frame is W, and the pixel height is H, the first sub- The size of the image is n*n as an example, and the number of the first sub-images is divided into N=WB×HB, where

Round up the operation. Optionally, in order to increase the speed of the processing, the size of the first sub-image may be set to 64*64.

After dividing the image frame into a plurality of first sub-images, the first calculation module 10 calculates a motion vector value of each of the first sub-images, and performs motion vector value pairs for each of the first sub-images. The plurality of first sub-images are classified, that is, a plurality of the first sub-images are classified according to a vector value interval in which the motion vector value of each of the first sub-images is located. Among them, the motion vector value can directly reflect the fluctuation strength of the image.

It should be noted that in video coding, the image content of the moving image adjacent to the image frame has time domain correlation. In this embodiment, the first calculating module 10 performs motion compensation on each of the first sub-images, searches for matching blocks of the first sub-image in adjacent image frames of the image frame, and records the first sub-image. Matching block motion vector MV _i,j =(Δx, Δy),

Is a motion vector value, where Δx represents the displacement of the first sub-image in the x-axis direction, and Δy represents the displacement of the first sub-image in the y-axis direction. For example, referring to FIG. 2, a first sub-image is selected in the image frame at time t, and a minimum distortion search is performed in the adjacent image frame at time t-1 to obtain a minimum distortion matching block, and the first first is calculated. The vector distance of the sub-image and the minimum distortion matching block, as shown in Fig. 2, MV = (Δx, Δy), thereby calculating the motion vector value of the selected first sub-image

Moreover, those skilled in the art can understand that the adjacent image frame of the currently selected image frame refers to the previous image frame on the frame sequence in which the image frame is selected.

After completing the classification of the first sub-image, the first calculation module 10 calculates a Peak Signal to Noise Ratio (PSNR) value of the first sub-image of each class. Where PSNR It is the most common and widely used objective standard for evaluating images.

The second calculating module 20 is configured to calculate an average value μ of the PSNR values of the first sub-images of each class, and calculate a standard deviation σ of the PSNR values of the first sub-images of each class, and preset a ratio of σ to μ Correct the fluctuation intensity of the code as the code after correction;

In this embodiment, the average value of the PSNR values of the first sub-images of each class

Standard deviation of PSNR values for the first sub-image of each class

Where k is an integer, each vector value interval corresponds to a k value, and P _k is a PSNR value of each type of first sub-image.

After calculating the σ value and the μ value of each type of sub-image, the second calculation module 20 corrects the ratio of σ to μ using the preset correction parameter φ as the encoded fluctuation intensity V, that is, the fluctuation intensity.

The φ value is an empirical parameter for normalizing the ratio of σ to μ to increase the processing speed. For example, the φ value is set to 100 in this embodiment.

In addition, in order to increase the processing speed, a maximum k value k _max may also be set, and a first sub-image with a motion vector value greater than k _max is divided into k _max intervals, where k _max is an empirical parameter, for example, this embodiment will k _{max is} set to 9.

The adjusting module 30 is configured to adjust the target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, where the target code rate of the encoding is adjusted according to the magnitude relationship between the fluctuation strength and the first preset threshold. : increasing the target code rate of the encoding when the fluctuation strength is greater than the first preset threshold;

When the fluctuation strength is less than the first predetermined threshold, the encoded target code rate is reduced.

It will be understood by those skilled in the art that in video coding, an image frame having a large fluctuation intensity requires more bits to be described than an image frame having a small fluctuation intensity. Therefore, in this embodiment, after the second calculation module 20 calculates the obtained coded fluctuation strength V, the adjustment module 30 determines whether the fluctuation intensity V is greater than a first preset threshold, and if the fluctuation strength V is greater than the first A predetermined threshold increases the encoded target rate; otherwise, if the fluctuation intensity V is less than or equal to the first predetermined threshold, the encoded target code rate is decreased. The first preset threshold is set according to actual needs. For example, in this embodiment, the first preset threshold is set to 5. In this embodiment, the step value TBR _{C of the} target rate adjustment is preset. The benchmark for each target rate adjustment.

For example, referring to FIG. 3, a measured view of the target code rate adjustment of the embodiment is provided. As shown in FIG. 3, in the encoding process, the target code rate is adjusted according to the change of the fluctuation intensity in real time. It is possible that the low bit occupancy occupies the visual decoding quality of the video image with minimal distortion.

Further, when the fluctuation intensity V is equal to the preset threshold, the encoded target code rate is not adjusted.

In other embodiments, a first preset threshold interval may be further set, when the fluctuation strength V is greater than a maximum value in the first preset threshold interval, increasing a target code rate of the encoding; when the fluctuation strength is When the V is located in the first preset threshold interval, the encoded target code rate is not adjusted; when the fluctuation strength V is smaller than the minimum value in the first preset threshold interval, the encoded target code rate is reduced. For example, the first preset interval is set to [4, 5].

The target rate adjustment apparatus of the present embodiment divides an image frame to be encoded into a plurality of first sub-images, and performs a plurality of the first sub-images according to motion vector values of each of the first sub-images. Classification, calculating the peak signal-to-noise ratio PSNR value of each type of first sub-image, and then comparing the standard deviation σ of the PSNR value of each type of the first sub-image with the average value μ as the encoded fluctuation intensity by using the preset correction parameter The resulting wave strength includes the reconstruction distortion of the image frame and the characteristics of motion compensation. When the fluctuation strength increases during the encoding process, based on the time domain correlation of the adjacent image frames, the next image frame to be encoded of the image frame to be encoded requires more bits for description (ie, encoding), therefore, Compared with the related art, the target code rate is selected by the imaginary value. The technical solution of the embodiment of the present invention can accurately reflect the bit required for the coding by adjusting the target code rate of the coded in real time by the fluctuation strength, and improve the selected target bit rate. Accuracy.

Embodiment 6

Optionally, based on the sixth embodiment, in the embodiment, the target rate adjustment apparatus further includes:

a recording module configured to record the currently calculated fluctuation intensity, and calculate an average value of the recorded fluctuation strength when the recorded fluctuation intensity reaches a second preset threshold;

An update module is arranged to update the encoded wave strength to the calculated average.

It can be understood by those skilled in the art that since the image content of the moving image adjacent to the image frame has time domain correlation, the image content of the adjacent consecutive multiple image frames tends to be substantially the same, correspondingly, the fluctuation intensity of the encoding in a short time. Changes are often negligible. That is, in practical applications, although each The code fluctuating intensity is calculated once for one frame, but the target bit rate is adjusted once for a long time, and there is a waste of processing resources. Therefore, in this embodiment, a window having a size of τ (the aforementioned second preset threshold, specifically, can be set as needed) is set, and the average fluctuation intensity of the image frame in the window is counted every τ frame to determine whether to perform The adjustment of the target bit rate.

In this embodiment, after completing the encoding of the image frame to be encoded and calculating the fluctuating intensity of the encoding, the recording module records the fluctuating intensity calculated at the time, and counts the calculated fluctuating intensity in the current window. Average fluctuation intensity of image frames

The update module updates the encoded wave strength to the calculated average wave strength

For the adjustment of the target code rate for the encoding by the adjustment module 30, reference may be made to the foregoing embodiments, and details are not described herein again. Where t is an integer multiple of τ, and t is used to indicate the position of the corresponding image frame in the sequence of frames.

In other embodiments, when recording the currently calculated fluctuation strength, the recording module may further smooth the currently calculated fluctuation intensity based on the last calculated fluctuation intensity to eliminate the influence of noise in the encoding process, in this embodiment. The recording module records the currently calculated fluctuation strength as: a·V _t-1 +b·V _t ;

Where a and b are empirical parameters (for example, in this embodiment, a takes a value of 0.1 and b takes a value of 0.9). V _t-1 represents the last calculated fluctuation intensity, and V _t represents the currently calculated fluctuation intensity.

In this embodiment, it is determined whether the target code rate needs to be adjusted and how to adjust by a certain interval, and the processing resource consumption is reduced on the basis of ensuring the accuracy of selecting the target code rate.

Example 7

Based on the fifth embodiment, referring to FIG. 7, in the embodiment, the first calculating module 10 includes:

The classification unit 101 is configured to divide the image frame to be encoded into a plurality of first sub-images, and calculate a motion vector value of each of the first sub-images of the image frame to be encoded relative to adjacent image frames, according to each Sorting the plurality of first sub-images by a vector value interval in which the motion vector value of the first sub-image is located;

A continuous video is composed of a series of image frames. For convenience of calculation, in the present embodiment, the classification unit 101 divides the image frame to be encoded into a plurality of first sub-images in a square shape. It can be understood by those skilled in the art that in other embodiments, the shape of the first sub-image may be selected according to actual needs. For example, the classification unit 101 may divide the image frame into a plurality of first sub-images that are rectangular. . In this embodiment, the number of the first sub-images divided depends on the size of the first sub-image, wherein the pixel width of the image frame is W, the pixel height is H, and the size of the first sub-image is n. *n is an example, the number of the first sub-images is divided into N=WB×HB, where

Round up the operation. Optionally, in order to increase the speed of the processing, the size of the first sub-image may be set to 64*64.

After dividing the image frame into a plurality of first sub-images, the classifying unit 101 calculates a motion vector value of each of the first sub-images, and pairs the motion vector values of each of the first sub-images The first sub-image is classified, that is, the plurality of first sub-images are classified according to a vector value interval in which the motion vector value of each of the first sub-images is located. Among them, those skilled in the art can understand that the motion vector value can directly reflect the fluctuation strength of the image.

It should be noted that in video coding, the image content of the moving image adjacent to the image frame has time domain correlation. In this embodiment, the classifying unit 101 performs motion compensation on each of the first sub-images, searches for matching blocks of the first sub-image in adjacent image frames of the image frame, and records matching blocks of the first sub-image. Motion vector MV _i,j =(Δx, Δy),

Is a motion vector value, where Δx represents the displacement of the first sub-image in the x-axis direction, and Δy represents the displacement of the first sub-image in the y-axis direction. For example, referring to FIG. 2, the classification unit 101 selects a first sub-image in the image frame at time t, performs minimum distortion search in the adjacent image frame at time t-1, obtains a minimum distortion matching block, and calculates The vector distance of the first sub-image and the minimum distortion matching block, as shown in FIG. 2, MV = (Δx, Δy), thereby calculating the motion vector value of the selected first sub-image

The calculated motion vector distribution of each first sub-image of the image frame is as shown in FIG.

The calculating unit 102 is configured to divide the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images according to the same division manner as the image frame to be encoded, and calculate each of the first sub-images And a mean square error between the second sub-images corresponding to each of the first sub-images; and calculating a PSNR value of each of the first sub-images based on each of the mean square errors.

It will be understood by those skilled in the art that in inter prediction coding, the next frame of coding is encoded based on the reconstructed frame of the current frame. In this embodiment, the calculating unit 102 divides the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images according to the same division manner as the image frame to be encoded, and calculates each of the first a mean square error between a sub-image and a second sub-image corresponding to each of the first sub-images, wherein a mean square error between the first sub-image and its corresponding second sub-image

Represents the standard deviation of the first sub-image,

Indicates the standard deviation of the second sub-image.

After calculating the mean square error between each of the first sub-images and the second sub-image corresponding to each of the first sub-images, the calculating unit 102 first counts the average distortion of the first sub-image of each class. D _k =∑D _i,j |k<||MV _i,j ||<k+1, where

For the motion vector size, k is an integer, and each of the aforementioned vector value intervals corresponds to a k value.

The computing unit 102 then calculates the PSNR value of each class first sub-image based on the average distortion of each class first sub-image, and the PSNR value of each class first sub-image.

Where bpp represents the number of bits when the pixel point gray value is expressed in binary, as in the 0 to 255 gray level representation, the value of bpp is 8.

Example eight

Based on any of the foregoing embodiments, in the embodiment, the target rate adjustment apparatus further includes a correction module configured to correct the adjusted target code rate based on the current network bandwidth to adopt the corrected target. The code rate is encoded.

It should be noted that the network bandwidth in this embodiment refers to the network bandwidth available for encoding video transmission, and those skilled in the art may understand that the network bandwidth is dynamically changed, and the network bandwidth is also local resources. The problem of preemption, therefore, in order to ensure the smoothness of the transmission of the encoded video, in this embodiment, the correction module acquires the currently available network bandwidth, and corrects the adjusted target code rate based on the acquired network bandwidth. , encoding with the corrected target bit rate.

Optionally, the modifying module corrects the adjusted target code rate based on the current network bandwidth by determining whether the current network bandwidth is greater than the adjusted target code rate; and in the current network. When the bandwidth is less than the adjusted target code rate, the network bandwidth is used as the corrected target code rate.

It is easy to understand that when the network bandwidth is smaller than the adjusted target code rate, The adjusted target bit rate is encoded, and the encoded video data cannot be smoothly transmitted to the target terminal, resulting in problems such as playing the card and affecting the user's visual experience. Therefore, in this embodiment, when the current network bandwidth is less than the adjusted target code rate, the correction module uses the network bandwidth as the corrected target code rate to ensure that the encoded video data can be Smooth transfer to the target terminal.

The embodiment of the invention further provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and the computer executable instructions are implemented to implement a target rate adjustment method.

The above is only an alternative embodiment of the present application, and thus does not limit the scope of the patent application, and the equivalent structure or equivalent process transformation of the specification and the drawings of the present application, or directly or indirectly applied to other related technologies. The fields are all included in the scope of patent protection of this application. One of ordinary skill in the art will appreciate that all or a portion of the above steps may be performed by a program to instruct related hardware, such as a processor, which may be stored in a computer readable storage medium, such as a read only memory, disk or optical disk. Wait. Alternatively, all or part of the steps of the above embodiments may also be implemented using one or more integrated circuits. Correspondingly, each module/unit in the above embodiment may be implemented in the form of hardware, for example, by implementing an integrated circuit to implement its corresponding function, or may be implemented in the form of a software function module, for example, executing a program stored in the memory by a processor. / instruction to achieve its corresponding function. This application is not limited to any specific combination of hardware and software. A person skilled in the art should understand that the technical solutions of the present application can be modified or equivalent, without departing from the spirit and scope of the technical solutions of the present application, and should be included in the scope of the claims of the present application.

Industrial applicability

The above technical solution can more accurately reflect the bits required for encoding, and improve the accuracy of the selected target bit rate.

Claims (10)

1. A target rate adjustment method, where the target rate adjustment method includes:

   Dividing the image frame to be encoded into a plurality of first sub-images, and classifying the plurality of first sub-images according to motion vector values of each of the first sub-images, and calculating peak values of each of the first sub-images Signal to noise ratio PSNR value;

   Calculating an average value μ of PSNR values of each type of first sub-image, and calculating a standard deviation σ of PSNR values of each type of first sub-image, and correcting the ratio of σ to μ as a coded fluctuation intensity by using a preset correction parameter;

   Adjusting the target code rate of the code according to the magnitude relationship between the fluctuation strength and the first preset threshold, wherein the adjusting the target code rate according to the magnitude relationship between the fluctuation strength and the first preset threshold includes:

   When the fluctuation strength is greater than the first preset threshold, increasing the encoded target code rate;

   When the fluctuation strength is less than the first predetermined threshold, the encoded target code rate is reduced.
2. The target rate adjustment method according to claim 1, before the step of adjusting the target code rate of the code according to the magnitude relationship between the fluctuation intensity and the first preset threshold, the method further includes:

   Recording the currently calculated fluctuation intensity, and calculating an average value of the recorded fluctuation strength when the recorded fluctuation intensity reaches the second preset threshold;

   The encoded wave strength is updated to the calculated average.
3. The target rate adjustment method according to claim 1, wherein the image frame to be encoded is divided into a plurality of first sub-images, and the motion vector values of each of the first sub-images are The first sub-images are classified, and the step of calculating the PSNR value of each type of first sub-image includes:

   Dividing the image frame to be encoded into a plurality of first sub-images, and calculating a motion vector value of each of the first sub-images of the image frame to be encoded relative to the adjacent image frames, according to each of the first sub- Sorting the plurality of first sub-images by a vector value interval in which a motion vector value of the image is located;

   Decomposing the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images in the same division manner as the image frame to be encoded, and calculating each of the first sub-images and each of the first a mean square error between the second sub-images corresponding to a sub-image;

   A PSNR value for each type of first sub-image is calculated based on each of the mean square errors.
4. The target rate adjustment method according to any one of claims 1 to 3, after the step of adjusting the target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, the method further includes:

   The adjusted target code rate is corrected based on the current network bandwidth to be encoded using the corrected target code rate.
5. The target rate adjustment method according to claim 4, wherein the step of correcting the adjusted target code rate based on the current network bandwidth comprises:

   Determining whether the current network bandwidth is greater than the adjusted target code rate;

   When the current network bandwidth is less than the adjusted target code rate, the network bandwidth is used as the corrected target code rate.
6. A target rate adjustment apparatus, the target rate adjustment apparatus comprising:

   a first calculating module, configured to divide the image frame to be encoded into a plurality of first sub-images, and classify the plurality of first sub-images according to motion vector values of each of the first sub-images, and calculate each Peak signal to noise ratio PSNR value of the first sub-image of the class;

   a second calculating module, configured to calculate an average value μ of PSNR values of each type of first sub-image, and calculate a standard deviation σ of PSNR values of each type of first sub-image, and correct the ratio of σ to μ by using a preset correction parameter After the wave strength as the code;

   The adjusting module is configured to adjust the target code rate of the encoding according to the magnitude relationship between the fluctuation strength and the first preset threshold, wherein the target code rate of the encoding is adjusted according to the magnitude relationship between the fluctuation strength and the first preset threshold include:

   When the fluctuation strength is greater than the first preset threshold, increasing the encoded target code rate;

   When the fluctuation strength is less than the first predetermined threshold, the encoded target code rate is reduced.
7. The target rate adjustment apparatus according to claim 6, wherein the target rate adjustment apparatus further comprises:

   a recording module configured to record the currently calculated fluctuation intensity, and calculate an average value of the recorded fluctuation strength when the recorded fluctuation intensity reaches a second preset threshold;

   An update module is arranged to update the encoded wave strength to the calculated average.
8. The target rate adjustment apparatus of claim 6, wherein the first calculation module comprises:

   a classifying unit configured to divide the image frame to be encoded into a plurality of first sub-images, and calculate a motion vector value of each of the first sub-images relative to the adjacent image frames of the image frame to be encoded, according to each Sorting the plurality of first sub-images by a vector value interval in which the motion vector value of the first sub-image is located;

   a calculating unit, configured to divide the reconstructed image frame of the image frame to be encoded into a plurality of second sub-images in the same division manner as the image frame to be encoded, and calculate each of the first sub-images and a mean square error between the second sub-images corresponding to each of the first sub-images; and calculating a PSNR value for each of the first sub-images based on each of the mean square errors.
9. The target rate adjustment apparatus according to any one of claims 6 to 8, wherein the target rate adjustment apparatus further comprises:

   The correction module is configured to correct the adjusted target code rate based on the current network bandwidth to encode using the corrected target code rate.
10. The target rate adjustment apparatus according to claim 9, wherein the correction module corrects the adjusted target code rate based on the current network bandwidth by:

    Determining whether the current network bandwidth is greater than the adjusted target code rate; and when the current network bandwidth is less than the adjusted target code rate, using the network bandwidth as the corrected target code rate.

Images