CN115866277A - Video encoding control method, video encoding control apparatus, and readable storage medium - Google Patents
Video encoding control method, video encoding control apparatus, and readable storage medium Download PDFInfo
- Publication number
- CN115866277A CN115866277A CN202211544547.9A CN202211544547A CN115866277A CN 115866277 A CN115866277 A CN 115866277A CN 202211544547 A CN202211544547 A CN 202211544547A CN 115866277 A CN115866277 A CN 115866277A
- Authority
- CN
- China
- Prior art keywords
- image
- video
- evaluation result
- quality evaluation
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The embodiment of the application discloses a video coding control method, video coding control equipment and a computer readable storage medium, which can control video coding under the condition of improving the coding quality of an image. The method of the embodiment of the application comprises the following steps: the method comprises the steps of obtaining a source image, obtaining a compressed image of a video to be coded, inputting the source image and the compressed image into a natural image quality evaluation NIQE model, obtaining a quality evaluation result of the compressed image, wherein the quality evaluation result represents the change condition of texture characteristics between the source image and the compressed image, and adjusting the coding quality parameter of the video to be coded according to the quality evaluation result, wherein the coding quality parameter is a parameter used for carrying out video coding control on the video to be coded.
Description
Technical Field
The embodiment of the application relates to the field of image quality evaluation, in particular to a video coding control method, a video coding control device and a computer readable storage medium.
Background
At present, in image (video) coding protocols such as H264/H265 and the like, the video coding protocol has good prediction stability, monotonicity and consistency and better consistency with subjective quality evaluation of human eyes, and is further connected to a video subjected to compression transmission for objective evaluation of image (video) quality, two indexes of Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM) are commonly used as evaluation standards of image quality, wherein the PSNR is a measuring method used for Signal reconstruction quality in the fields of image compression and the like, and the coding quality parameter of the video to be coded is adjusted according to the values calculated by the PSNR and the SSIM so as to perform video coding control on the video to be coded according to the coding quality parameter.
However, in recent years, it has been found that high PSNR and SSIM do not necessarily represent high quality image quality, for example, there is an image with a background of pure blue and a text in the image, and after the image is subjected to lossy transmission compression from a lossless image, the background image of pure blue is locally changed, and the image is changed from deep blue to light blue as a whole, but the image may not be seen by human eyes. For that character, after a local change is made, the character may become blurry and become less clear, but since the ratio of the character in the whole image is small, it is calculated based on PSNR and SSIM, and it is considered that the structural similarity of the image has not particularly large change, so that the character seen by human eyes becomes very blurred from the edge and texture, the definition of the image is low, but the change of the value calculated based on PSNR and SSIM is not very large, and it is concluded that the definition of the image is high, which is contrary to the conclusion of human eyes. Therefore, the PSNR and SSIM based video coding control methods result in poor image coding quality.
Disclosure of Invention
The embodiment of the application provides a video coding control method, a video coding control device and a computer readable storage medium, which can control video coding under the condition of improving the coding quality of an image.
In a first aspect, an embodiment of the present application provides an image quality evaluation method, including:
obtaining a source image;
obtaining a compressed image of a video to be coded;
inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model to obtain a quality evaluation result of the compressed image, wherein the quality evaluation result represents the change condition of texture characteristics between the source image and the compressed image;
and adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result, wherein the encoding quality parameter is a parameter used for carrying out video encoding control on the video to be encoded.
Optionally, the obtaining a quality evaluation result of the compressed image includes:
screening out a first target image block with the sharpness value larger than or equal to a preset sharpness threshold in the source image, and determining a first spatial domain characteristic value of each pixel point in the first target image block and a second spatial domain characteristic value of each pixel point in the second target image block after screening out a second target image block with the sharpness value larger than or equal to the preset sharpness threshold in the compressed image;
obtaining a first multivariate Gaussian model of the source image according to a first airspace characteristic value of each pixel point in the first target image block, and obtaining a second multivariate Gaussian model of the source image according to a second airspace characteristic value of each pixel point in the second target image block;
and obtaining the distance between the first multivariate Gaussian model and the second multivariate Gaussian model, and taking the distance as the quality evaluation result of the compressed image.
Optionally, the encoding quality parameter includes a quantization parameter QP.
Optionally, before obtaining the source image, the method further includes:
determining a target coding quality parameter value corresponding to a target quality evaluation result;
optionally, the adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result includes:
and if the quality evaluation result of the video to be coded is not within the range of the target quality evaluation result, adjusting the coding quality parameter of the video to be coded to the target coding quality parameter value.
Optionally, the determining a target coding quality parameter value corresponding to the target quality evaluation result includes:
after a source image sample and a compressed image sample of a video sample to be coded are obtained, inputting the source image sample and the compressed image sample into the NIQE model, and obtaining a quality evaluation result sample of the compressed image sample;
fine-tuning the coding quality parameter corresponding to the quality evaluation result sample for a preset number of times according to a preset amplitude to obtain the fluctuation condition of the quality evaluation result sample during each fine-tuning;
and if the fluctuation condition meets the preset fluctuation condition, determining the coding quality parameter value sample corresponding to the fluctuation condition as the target coding quality parameter value.
Optionally, the adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result includes:
if the quality evaluation result is not in the range of the target quality evaluation result, carrying out fine adjustment on the coding quality parameter corresponding to the quality evaluation result for a preset number of times according to a preset amplitude to obtain the fluctuation condition of the quality evaluation result during each fine adjustment;
determining a target coding quality parameter; wherein the fluctuation condition corresponding to the target coding quality parameter meets a preset fluctuation condition;
and adjusting the encoding quality parameter of the video to be encoded to the target encoding quality parameter.
Optionally, the source image is a locally preset image.
Optionally, the source image is an image of a video to be encoded.
In a second aspect, an embodiment of the present application provides a video coding control device, including:
an obtaining unit for obtaining a source image;
the obtaining unit is further configured to obtain a compressed image of the video to be encoded;
the input unit is used for inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model to obtain a quality evaluation result of the compressed image, and the quality evaluation result represents the change condition of texture features between the source image and the compressed image;
and the adjusting unit is used for adjusting the coding quality parameter of the video to be coded according to the quality evaluation result, wherein the coding quality parameter is a parameter used for carrying out video coding control on the video to be coded.
In a third aspect, an embodiment of the present application provides a video coding control device, including:
the system comprises a central processing unit, a memory, an input/output interface, a wired or wireless network interface and a power supply;
the memory is a transient memory or a persistent memory;
the central processor is configured to communicate with the memory and execute the instruction operations in the memory to execute the aforementioned video coding control method.
In a fourth aspect, the present application provides a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to execute the aforementioned video coding control method.
In a fifth aspect, the present application provides a computer program product containing instructions, which when run on a computer, cause the computer to execute the aforementioned video coding control method. According to the technical scheme, the embodiment of the application has the following advantages: the method comprises the steps of obtaining a source image, obtaining a compressed image of a video to be coded, inputting the source image and the compressed image into a natural image quality evaluation NIQE model, obtaining a quality evaluation result of the compressed image, wherein the quality evaluation result represents the change condition of texture characteristics between the source image and the compressed image, adjusting the coding quality parameter of the video to be coded according to the quality evaluation result, the coding quality parameter is a parameter used for performing video coding control on the video to be coded, the characteristics of texture details can be calculated, the accuracy of the image quality evaluation result is high, the coding quality parameter with the best effect can be adjusted based on the image quality evaluation result with the high accuracy so as to perform video coding control on the video to be coded with the best effect, and the coding quality of the obtained image is good.
Drawings
Fig. 1 is a schematic diagram of an architecture of a video coding control system according to an embodiment of the present application;
fig. 2 is a flowchart illustrating a video encoding control method according to an embodiment of the present application;
FIG. 3 is a functional mapping relationship between NIQE values and QP values disclosed in the embodiments of the present application;
fig. 4 is a schematic structural diagram of a video encoding control apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of another video encoding control apparatus disclosed in an embodiment of the present application;
fig. 6 is a schematic structural diagram of another video encoding control apparatus according to an embodiment of the present disclosure.
Detailed Description
The embodiment of the application provides a video coding control method, a video coding control device and a computer readable storage medium, which can control video coding under the condition of improving the coding quality of an image. .
Referring to fig. 1, an architecture of a video encoding control system 100 according to an embodiment of the present application includes:
a server 101 and a client 102. When performing video encoding control, the server 101 may be connected to the client 102. The server 101 can obtain a video to be encoded, the server 101 can compress the video to be encoded according to the encoding quality parameters to obtain a compressed image of the video to be encoded, the compressed image of the video to be encoded can be sent to the client 102, the client 102 can obtain a source image, and can obtain the compressed image of the video to be encoded sent by the server 101, the quality evaluation can be performed on the compressed image according to the source image and the compressed image of the video to be encoded to obtain a quality evaluation result of the compressed image, the encoding quality parameters of the video to be encoded can be adjusted according to the quality evaluation result, and the video to be encoded can be subjected to video encoding control according to the encoding quality parameters.
Referring to fig. 2 based on the video coding control system shown in fig. 1, fig. 2 is a schematic flowchart of a video coding control method disclosed in an embodiment of the present application, where the method includes:
201. a source image is obtained.
In this embodiment, when video encoding control is performed, a source image can be obtained.
202. A compressed image of the video to be encoded is obtained.
After the source image is obtained, a compressed image of the video to be encoded may be obtained.
203. And inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model to obtain a quality evaluation result of the compressed image, wherein the quality evaluation result represents the change condition of texture characteristics between the source image and the compressed image.
After obtaining the compressed image of the video to be encoded, the source image and the compressed image can be input into a Natural Image Quality Evaluation (NIQE) model, and the quality evaluation result of the compressed image is obtained, wherein the quality evaluation result represents the change condition of the texture characteristics between the source image and the compressed image.
204. And adjusting the coding quality parameter of the video to be coded according to the quality evaluation result, wherein the coding quality parameter is used for carrying out video coding control on the video to be coded.
And inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model, and after obtaining the quality evaluation result of the compressed image, adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result, wherein the encoding quality parameter is a parameter used for performing video encoding control on the video to be encoded. The method for adjusting the coding quality parameter of the video to be coded according to the quality evaluation result may be to determine a target coding quality parameter value corresponding to the target quality evaluation result, and adjust the coding quality parameter of the video to be coded to the target coding quality parameter value if the quality evaluation result of the video to be coded is not within the range of the target quality evaluation result, or may be other reasonable methods, and is not limited herein.
In the embodiment of the application, a source image can be obtained, a compressed image of a video to be coded is obtained, the source image and the compressed image are input into a natural image quality evaluation NIQE model, a quality evaluation result of the compressed image is obtained, the quality evaluation result represents the change situation of texture characteristics between the source image and the compressed image, a coding quality parameter of the video to be coded is adjusted according to the quality evaluation result, the coding quality parameter is a parameter used for performing video coding control on the video to be coded, the characteristics of texture details can be calculated, the accuracy of the image quality evaluation result is high, the coding quality parameter with the best effect can be adjusted based on the image quality evaluation result with high accuracy so as to perform video coding control with the best effect on the video to be coded, and the coding quality of the obtained image is good.
In the embodiment of the present application, there may be a plurality of methods for adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result, and one of the methods is described below based on the video encoding control method shown in fig. 2.
In this embodiment, when video encoding control is performed, a source image can be obtained. Specifically, the source image may be a locally preset image, or may be an image of a video to be encoded.
A compressed image of the video to be encoded may be obtained. Specifically, the compressed image of the video to be encoded sent by the server may be received after the server compresses the image of the video to be encoded to obtain the compressed image of the video to be encoded, or may be a client (image quality evaluation device) that compresses the image of the video to be encoded to obtain the compressed image of the video to be encoded, or may be other reasonable devices or electronic units that can obtain the compressed image of the video to be encoded, which is not limited herein, and the specific method for obtaining the compressed image of the video to be encoded is not limited herein.
After the source Image and the compressed Image are obtained, the source Image and the compressed Image can be input into a Natural Image Quality Evaluation (NIQE) model, and a Quality evaluation result of the compressed Image is obtained, wherein the Quality evaluation result represents the variation condition of texture features between the source Image and the compressed Image.
The method for obtaining the quality evaluation result of the compressed image can be that after a first target image block with the sharpness value larger than or equal to a preset sharpness threshold value in a source image and a second target image block with the sharpness value larger than or equal to the preset sharpness threshold value in the compressed image are screened out, a first airspace characteristic value of each pixel point in the first target image block and a second airspace characteristic value of each pixel point in the second target image block are firstly determined; then obtaining a first multivariate Gaussian model of the source image according to the first airspace characteristic value of each pixel point in the first target image block, and obtaining a second multivariate Gaussian model of the source image according to the second airspace characteristic value of each pixel point in the second target image block; and finally, obtaining the distance between the first multivariate Gaussian model and the second multivariate Gaussian model, and taking the distance as the quality evaluation result of the compressed image. It can be understood that the sharpness variation of the image can be measured by extracting spatial domain features (similarity variation of pixels between images) of the source image and the lossy image (compressed image) and calculating.
The method for screening out the first target image block with the sharpness value larger than or equal to the preset sharpness threshold value in the source image and the second target image block with the sharpness value larger than or equal to the preset sharpness threshold value in the compressed image can be that the source image and the compressed image are divided into at least one image block according to the preset size, then the first sharpness value of each image block in the source image and the second sharpness value of each image block in the compressed image are determined, whether the first sharpness value of the image block and the second sharpness value of the image block are larger than or equal to the preset sharpness threshold value or not is judged, if the first sharpness value of the image block is larger than or equal to the preset sharpness threshold value, the image block corresponding to the first sharpness value is determined to be the first target image block, and if the second sharpness value of the image block is larger than or equal to the preset sharpness threshold value, the image block corresponding to the second sharpness value is determined to be the second target image block.
The method for determining the first sharpness value of each image block in the source image and the second sharpness value of each image block in the compressed image may be that, first, gaussian blur processing is performed on each image block in the source image and each image block in the compressed image respectively to obtain a local standard deviation of each pixel point of each image block in the source image and a local standard deviation of each pixel point of each image block in the compressed image, then, for each image block of the source image, the sum of the local standard deviations corresponding to each pixel point in the image block is used as the first sharpness value of the image block, and for each image block of the compressed image, the sum of the local standard deviations corresponding to each pixel point in the image block is used as the second sharpness value of the image block.
The method for respectively carrying out Gaussian blur processing on each image block in a source image and each image block in a compressed image to obtain the local standard deviation of each pixel point of each image block in the source image and the local standard deviation of each pixel point of each image block in the compressed image can be that firstly, the Gaussian blur processing is respectively carried out on each pixel point of each image block in the source image and each pixel point of each image block in the compressed image to obtain the local mean value of each pixel point of each image block in the source image and the local mean value of each image block in the compressed image, then, the Gaussian blur processing is carried out on the square of the difference between the pixel value of each pixel point of each image block in the source image and the local mean value to obtain the local standard deviation of each pixel point of each image block in the source image, and the Gaussian blur processing is carried out on the square of the difference between the pixel value of each pixel point of each image block in the compressed image and the local mean value to obtain the local standard deviation of each pixel point of each image block in the compressed image.
Specifically, for the source image and the compressed image, the method for calculating the local mean value of each pixel point of each image block, the method for calculating the local standard deviation of each pixel point of each image block, and the method for calculating the first sharpness value and the second sharpness value may be as follows:
the first formula represents that Gaussian blur processing is performed on each pixel point of each image block to obtain a local mean value of each pixel point of each image block, the second formula represents that Gaussian blur processing is performed on the square of the difference value between the pixel value of each pixel point of each image block and the local mean value to obtain a local standard deviation of each pixel point of each image block, and the first formula and the second formula are { omega [ omega ] in the first formula and the second formula k,l K = -K.. K; l = -L.. L } is a two-dimensional circularly symmetric gaussian weight function sampled from 3 standard deviations (K = L = 3) and re-scaled to a unit volume, and formula three represents a sum of local standard deviations corresponding to each pixel point in an image block as a first sharpness value of each image block for each image block of the compressed image or a sum of local standard deviations corresponding to each pixel point in an image block as a second sharpness value of an image block for each image block of the compressed image, wherein patch B represents an image block that has been divided into P × P, δ represents a sharpness value of an image block, the sharpness values of the local images are quantized using the local standard deviations, the image block of P × P is labeled with B =1,2,..., B, and an average local offset range of each image block is calculated using a direct method.
After a first target image block of a source image and a second target image block of a compressed image are screened out, a first airspace characteristic value of each pixel point in the first target image block and a second airspace characteristic value of each pixel point in the second target image block can be calculated.
Specifically, the first airspace feature value of each pixel point in the first target image block can be obtained according to the local mean value and the local standard deviation of each pixel point in the first target image block, and the second airspace feature value of each pixel point in the second target image block can be obtained according to the local mean value and the local standard deviation of each pixel point in the second target image block. Specifically, the calculation method of the first spatial domain eigenvalue of each pixel point in the first target image block and the second spatial domain eigenvalue of each pixel point in the second target image block may be:
in the formula IV, i belongs to {1,2.. M }, j belongs to {1,2.. N } is the space coordinate of the image, and M and N are the space coordinates of the image
^
Dimension, I (I, j) is a first spatial domain feature value of each pixel point in the first target image block or a second spatial domain feature value of each pixel point in the second target image block, I (I, j) is a pixel value of each pixel point in the first target image block or a pixel value of each pixel point in the second target image block, μ (I, j) is a local mean value of each pixel point in the first target image block or a local mean value of each pixel point in the second target image block, and σ (I, j) is a local standard deviation of each pixel point in the first target image block or a local standard deviation of each pixel point in the second target image block.
The method for determining the first spatial domain characteristic value of each pixel point in the first target image block and the second spatial domain characteristic value of each pixel point in the second target image block may be to obtain the first spatial domain characteristic value of each pixel point in the first target image block according to the local mean value and the local standard deviation of each pixel point in the first target image block, and obtain the second spatial domain characteristic value of each pixel point in the second target image block according to the local mean value and the local standard deviation of each pixel point in the second target image block. Specifically, the calculation method of the first spatial domain eigenvalue of each pixel point in the first target image block and the second spatial domain eigenvalue of each pixel point in the second target image block may be:
in the fourth formula, i is from {1,2,. M }, j is from {1,2,. N } is the space coordinate of the image, M and N are the image space coordinates
^
Dimension, I (I, j) is a first spatial domain feature value of each pixel point in the first target image block or a second spatial domain feature value of each pixel point in the second target image block, I (I, j) is a pixel value of each pixel point in the first target image block or a pixel value of each pixel point in the second target image block, μ (I, j) is a local mean value of each pixel point in the first target image block or a local mean value of each pixel point in the second target image block, and σ (I, j) is a local standard deviation of each pixel point in the first target image block or a local standard deviation of each pixel point in the second target image block.
The method for obtaining the distance between the first multivariate gaussian model and the second multivariate gaussian model may be that a first mean vector and a first covariance matrix of the first multivariate gaussian model are estimated according to a maximum likelihood estimation algorithm, a second mean vector and a second covariance matrix of the second multivariate gaussian model are estimated according to the maximum likelihood estimation algorithm, then a target distance between the first mean vector and the first covariance matrix and between the second mean vector and the second covariance matrix is calculated, and the target distance is used as the distance between the first multivariate gaussian model and the second multivariate gaussian model.
It can be understood that the calculation method of the first multivariate gaussian model fitting the source image according to the first spatial domain characteristic value of each pixel point in the first target image block and the second multivariate gaussian model fitting the source image according to the second spatial domain characteristic value of each pixel point in the second target image block may be:
in the formula V, (x) 1 ,...,x k ) Is a first spatial domain eigenvalue or a second spatial domain eigenvalue, v represents a first mean vector of a first multivariate Gaussian Model (MVG) or a second mean vector of a second multivariate Gaussian Model (MVG), ΣA first covariance matrix representing a first mean vector of a first multivariate Gaussian Model (MVG) or a second covariance matrix of a second multivariate Gaussian Model (MVG).
The calculation method of calculating the target distance between the first mean vector and the first covariance matrix and the second mean vector and the second covariance matrix may be:
in the sixth formula, v 1 ,v 2 ,∑ 1 ,∑ 2 Respectively representing a first mean vector, a second mean vector, a first covariance matrix and a second covariance matrix, D (v) 1 ,v 2 ,∑ 1 ,∑ 2 ) Represents the target distance, (v) 1 ,v 2 ) T A matrix transpose representing the first mean vector and the second mean vector.
And inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model, and after obtaining the quality evaluation result of the compressed image, adjusting the coding quality parameter of the video to be coded according to the quality evaluation result, wherein the coding quality parameter is a parameter for controlling video coding of the video to be coded. The encoding quality parameters may include Quantization Parameters (QP), frame rate, image size, resolution, and other parameters that affect the size of the encoded bitstream.
It is worth mentioning that before obtaining the source image, a target coding quality parameter value corresponding to the target quality evaluation result may be determined. Specifically, the method for determining the target coding quality parameter value corresponding to the target quality evaluation result may be that, after obtaining a source image sample and a compressed image sample of a video sample to be coded, the source image sample and the compressed image sample are input into an NIQE model to obtain a quality evaluation result sample of the compressed image sample, then the coding quality parameter corresponding to the quality evaluation result sample is fine-tuned for a preset number of times according to a preset amplitude to obtain a fluctuation condition of the quality evaluation result sample during each fine-tuning, and if the fluctuation condition meets a preset fluctuation condition, the coding quality parameter value sample corresponding to the fluctuation condition is determined as the target coding quality parameter value. Specifically, the method for fine-tuning the coding quality parameter corresponding to the quality evaluation result sample by the preset number of times according to the preset amplitude may be that the coding quality parameter corresponding to the quality evaluation result sample is increased by the preset number of times according to the preset amplitude, the fluctuation condition of the quality evaluation result sample increase at each increase is calculated, and if the fluctuation condition is a fluctuation condition that is suddenly reduced, the coding quality parameter value sample corresponding to the fluctuation condition is determined as the target coding quality parameter value. The coding quality parameter corresponding to the quality evaluation result sample may also be reduced by a preset number of times according to a preset amplitude, the fluctuation condition of the quality evaluation result sample reduced in each reduction is calculated, and if the fluctuation condition is a suddenly increased fluctuation condition, the coding quality parameter value sample corresponding to the fluctuation condition is determined as the target coding quality parameter value, and other reasonable fine-tuning methods may also be used, which are not limited herein.
After the target coding quality parameter value corresponding to the target quality evaluation result is determined, the method for adjusting the coding quality parameter of the video to be coded according to the quality evaluation result may be that if the quality evaluation result of the video to be coded is not within the range of the target quality evaluation result, the coding quality parameter of the video to be coded is adjusted to the target coding quality parameter value.
The method for adjusting the coding quality parameters of the video to be coded according to the quality evaluation result can also be that if the quality evaluation result is not within the range of the target quality evaluation result, the coding quality parameters corresponding to the quality evaluation result are finely adjusted for a preset number of times according to a preset amplitude to obtain the fluctuation condition of the quality evaluation result during each fine adjustment, and then the target coding quality parameters are determined; and finally, adjusting the coding quality parameter of the video to be coded to the target coding quality parameter. Specifically, the coding quality parameter corresponding to the quality evaluation result may be increased by a preset number of times according to a preset amplitude, so as to obtain a fluctuation condition of the quality evaluation result in each increase, determine a target coding quality parameter of which the fluctuation condition is suddenly reduced, and adjust the coding quality parameter of the video to be coded to the target coding quality parameter. The encoding quality parameter corresponding to the quality evaluation result may be reduced for a preset number of times according to a preset amplitude, so as to obtain a fluctuation condition of the quality evaluation result at each time of increase, determine a target encoding quality parameter of which the fluctuation condition is an abruptly increased fluctuation condition, and adjust the encoding quality parameter of the video to be encoded to the target encoding quality parameter, or other reasonable fine-tuning methods, which are not limited herein.
Referring to fig. 3 in particular, fig. 3 is a functional mapping relationship between an NIQE value and a QP value according to an embodiment of the present disclosure. In fig. 3, as the QP value decreases, the NIQE value also decreases continuously, and at a certain QP position, the decrease of the NIQE value tends to be gradual, that is, the QP value at that position is the optimal QP position. The QP value decreases again and there is no significant increase in quality for the human eye, but the bandwidth consumption for the network continues to increase.
After confirming the best QP value, the user can be provided with the best viewing experience in the equivalent network state. Because the target code rate is mainly output by adjusting the size of the QP value in the existing code rate control algorithm, the QP value reflects the compression condition of the image or video space details, if the QP value is smaller, most of the details are reserved, but the network bandwidth consumption is huge, if the QP value is increased, the network bandwidth consumption is reduced, but the image local details are lost, the image distortion is enhanced and the quality is reduced, the confirmation of the optimal QP value is beneficial to obtaining better balance on the image quality and the network bandwidth consumption, wherein the image quality can directly influence the user experience.
It can be understood that, by setting QP values for HEVC image (video) codec through the NIQE values, a relation graph of QP values and NIQE values can be constructed, if the NIQE values are used as a measure, the NIQE values are a non-linear value decreasing trend as the QP values decrease, and if PSNR and SSIM are used as measures, the QP values may have a trend of local value becoming larger. Based on the method, the same QP value is used as quality evaluation, the images with the same quality are extracted and sent to an NIQE model for NIQE value calculation, and the difference of numerical values can be about 0.1%.
It is worth mentioning that when the source image is a locally preset image, the source image of the video to be encoded does not need to be referred to, and compared with the prior art, the data size is reduced and the data transmission efficiency is improved under the condition that the accuracy of image quality evaluation is ensured. When the source image is the image of the video to be coded, compared with the prior art, although the source image of the video to be coded is referred to, the image quality evaluation accuracy is higher.
It is to be understood that, in addition to the above-described method of compressing the quality evaluation result of an image; in addition to the above described method of adjusting the encoding quality parameters of the video to be encoded according to the quality evaluation results; in addition to the above-described method of determining a target encoding quality parameter value corresponding to a target quality evaluation result; in addition to the above-described method of screening out a first target image block having a sharpness value greater than or equal to a preset sharpness threshold in a source image, and compressing a second target image block having a sharpness value greater than or equal to a preset sharpness threshold in an image; except for the above-described method of determining the first spatial domain eigenvalue of each pixel point in the first target image block and the second spatial domain eigenvalue of each pixel point in the second target image block; in addition to the above described method of obtaining the distance of the first multivariate gaussian model from the second multivariate gaussian model; other reasonable methods are also possible, and are not limited herein.
In the embodiment, a source image can be obtained, a compressed image of a video to be encoded is obtained, the source image and the compressed image are input into a Natural Image Quality Evaluation (NIQE) model, a quality evaluation result of the compressed image is obtained, the quality evaluation result represents the change condition of texture characteristics between the source image and the compressed image, the encoding quality parameter of the video to be encoded is adjusted according to the quality evaluation result, the encoding quality parameter is a parameter used for performing video encoding control on the video to be encoded, the characteristics of texture details can be calculated, the accuracy of the image quality evaluation result is high, the encoding quality parameter with the best effect can be adjusted based on the image quality evaluation result with high accuracy so as to perform video encoding control with the best effect on the video to be encoded, and the encoding quality of the obtained image is good. Moreover, the encoding parameters can be adjusted according to the quality evaluation result, the optimal image (video) service can be provided for the client, the image (video) quality can be ensured while the compression rate is ensured, the satisfaction degree of the client is improved, and the loss of the client is reduced.
With reference to fig. 4, the video coding control method in the embodiment of the present application is described above, and the video coding control device in the embodiment of the present application is described below, where an embodiment of the video coding control device in the embodiment of the present application includes:
an obtaining unit 401 for obtaining a source image;
the obtaining unit 401 is further configured to obtain a compressed image of a video to be encoded;
an input unit 402, configured to input the source image and the compressed image obtained by the obtaining unit 401 into a natural image quality evaluation NIQE model, and obtain a quality evaluation result of the compressed image, where the quality evaluation result represents a variation condition of texture features between the source image and the compressed image;
an adjusting unit 403, configured to adjust a coding quality parameter of the video to be coded according to the quality evaluation result obtained by the input unit 402, where the coding quality parameter is a parameter used for performing video coding control on the video to be coded.
Referring to fig. 5, a video encoding control device in an embodiment of the present application is described in detail below, where another embodiment of the video encoding control device in the embodiment of the present application includes:
an obtaining unit 501, configured to obtain a source image;
the obtaining unit 501 is further configured to obtain a compressed image of a video to be encoded;
an input unit 502, configured to input the source image and the compressed image obtained by the obtaining unit 501 into a natural image quality evaluation NIQE model, and obtain a quality evaluation result of the compressed image, where the quality evaluation result represents a change situation of texture features between the source image and the compressed image;
an adjusting unit 503, configured to adjust a coding quality parameter of the video to be coded according to the quality evaluation result obtained by the input unit 502, where the coding quality parameter is a parameter used for performing video coding control on the video to be coded.
The obtaining unit 501 is specifically configured to screen out a first target image block in the source image whose sharpness value is greater than or equal to a preset sharpness threshold, and a second target image block in the compressed image whose sharpness value is greater than or equal to the preset sharpness threshold, determine a first spatial feature value of each pixel in the first target image block and a second spatial feature value of each pixel in the second target image block, obtain a first multivariate gaussian model of the source image according to the first spatial feature value of each pixel in the first target image block, obtain a second multivariate gaussian model of the source image according to the second spatial feature value of each pixel in the second target image block, obtain a distance between the first multivariate gaussian model and the second multivariate gaussian model, and use the distance as a quality evaluation result of the compressed image.
The video encoding control apparatus further includes:
a determining unit 504, configured to determine a target coding quality parameter value corresponding to the target quality evaluation result;
the adjusting unit 503 is specifically configured to adjust the encoding quality parameter of the video to be encoded to the target encoding quality parameter value if the quality evaluation result of the video to be encoded is not within the range of the target quality evaluation result.
The determining unit 504 is specifically configured to, after obtaining a compressed image sample of a source image sample and a video sample to be encoded, input the source image sample and the compressed image sample into the NIQE model, obtain a quality evaluation result sample of the compressed image sample, perform fine adjustment for a preset number of times on a coding quality parameter corresponding to the quality evaluation result sample according to a preset amplitude, obtain a fluctuation condition of the quality evaluation result sample during each fine adjustment, and if the fluctuation condition satisfies a preset fluctuation condition, determine the coding quality parameter value sample corresponding to the fluctuation condition as the target coding quality parameter value.
The adjusting unit 503 is specifically configured to, if the quality evaluation result is not within the range of the target quality evaluation result, perform fine adjustment on the coding quality parameter corresponding to the quality evaluation result by a preset number of times according to a preset amplitude, obtain a fluctuation condition of the quality evaluation result during each fine adjustment, and determine the target coding quality parameter; and adjusting the coding quality parameter of the video to be coded to the target coding quality parameter when the fluctuation condition corresponding to the target coding quality parameter meets a preset fluctuation condition.
Referring to fig. 6, another embodiment of the video encoding control apparatus 600 according to the embodiment of the present application includes:
a central processing unit 601, a memory 605, an input/output interface 604, a wired or wireless network interface 603 and a power supply 602;
the memory 605 is a transient storage memory or a persistent storage memory;
the central processor 601 is configured to communicate with the memory 605 and execute the operations of the instructions in the memory 605 to perform the methods described in the embodiments of fig. 2.
The embodiment of the present application further provides a computer-readable storage medium, which includes instructions, when the instructions are executed on a computer, cause the computer to execute the method in the foregoing embodiment shown in fig. 2.
The embodiment of the present application also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the method in the foregoing embodiment shown in fig. 2.
It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, which are essential or part of the technical solutions contributing to the prior art, or all or part of the technical solutions, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Claims (11)
1. A video encoding control method, comprising:
obtaining a source image;
obtaining a compressed image of a video to be coded;
inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model to obtain a quality evaluation result of the compressed image, wherein the quality evaluation result represents the change condition of texture characteristics between the source image and the compressed image;
and adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result, wherein the encoding quality parameter is a parameter used for carrying out video encoding control on the video to be encoded.
2. The method according to claim 1, wherein the obtaining of the quality evaluation result of the compressed image comprises:
screening out a first target image block with a sharpness value larger than or equal to a preset sharpness threshold value in the source image and a second target image block with a sharpness value larger than or equal to the preset sharpness threshold value in the compressed image, and then determining a first spatial characteristic value of each pixel point in the first target image block and a second spatial characteristic value of each pixel point in the second target image block;
obtaining a first multivariate Gaussian model of the source image according to a first airspace characteristic value of each pixel point in the first target image block, and obtaining a second multivariate Gaussian model of the source image according to a second airspace characteristic value of each pixel point in the second target image block;
and obtaining the distance between the first multivariate Gaussian model and the second multivariate Gaussian model, and taking the distance as the quality evaluation result of the compressed image.
3. The method of claim 1, wherein the encoding quality parameter comprises a Quantization Parameter (QP).
4. The method according to claim 1 or 3, characterized in that before said obtaining a source image, the method further comprises:
determining a target coding quality parameter value corresponding to a target quality evaluation result;
the adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result comprises:
and if the quality evaluation result of the video to be coded is not within the range of the target quality evaluation result, adjusting the coding quality parameter of the video to be coded to the target coding quality parameter value.
5. The method according to claim 4, wherein the determining a target encoding quality parameter value corresponding to the target quality evaluation result comprises:
after a source image sample and a compressed image sample of a video sample to be coded are obtained, inputting the source image sample and the compressed image sample into the NIQE model, and obtaining a quality evaluation result sample of the compressed image sample;
fine-tuning the coding quality parameter corresponding to the quality evaluation result sample for a preset number of times according to a preset amplitude to obtain the fluctuation condition of the quality evaluation result sample during each fine-tuning;
and if the fluctuation condition meets the preset fluctuation condition, determining the coding quality parameter value sample corresponding to the fluctuation condition as the target coding quality parameter value.
6. The method according to claim 1, wherein said adjusting the encoding quality parameter of the video to be encoded according to the quality evaluation result comprises:
if the quality evaluation result is not in the range of the target quality evaluation result, carrying out fine adjustment on the coding quality parameter corresponding to the quality evaluation result for a preset number of times according to a preset amplitude to obtain the fluctuation condition of the quality evaluation result during each fine adjustment;
determining a target coding quality parameter; wherein the fluctuation condition corresponding to the target coding quality parameter meets a preset fluctuation condition;
and adjusting the encoding quality parameter of the video to be encoded to the target encoding quality parameter.
7. Method according to claim 2, characterized in that the source image is a locally preset image.
8. Method according to claim 2, characterized in that said source images are images of a video to be encoded.
9. A video encoding control apparatus characterized by comprising:
an obtaining unit for obtaining a source image;
the obtaining unit is further configured to obtain a compressed image of the video to be encoded;
the input unit is used for inputting the source image and the compressed image into a Natural Image Quality Evaluation (NIQE) model to obtain a quality evaluation result of the compressed image, and the quality evaluation result represents the change condition of texture features between the source image and the compressed image;
and the adjusting unit is used for adjusting the coding quality parameter of the video to be coded according to the quality evaluation result, wherein the coding quality parameter is a parameter used for carrying out video coding control on the video to be coded.
10. A video encoding control apparatus characterized by comprising:
the system comprises a central processing unit, a memory, an input/output interface, a wired or wireless network interface and a power supply;
the memory is a transient memory or a persistent memory;
the central processor is configured to communicate with the memory and execute the operations of the instructions in the memory to perform the method of any of claims 1 to 8.
11. A computer-readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211544547.9A CN115866277A (en) | 2022-11-30 | 2022-11-30 | Video encoding control method, video encoding control apparatus, and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211544547.9A CN115866277A (en) | 2022-11-30 | 2022-11-30 | Video encoding control method, video encoding control apparatus, and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115866277A true CN115866277A (en) | 2023-03-28 |
Family
ID=85669658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211544547.9A Pending CN115866277A (en) | 2022-11-30 | 2022-11-30 | Video encoding control method, video encoding control apparatus, and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115866277A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116800976A (en) * | 2023-07-17 | 2023-09-22 | 武汉星巡智能科技有限公司 | Audio and video compression and restoration method, device and equipment for infant with sleep |
-
2022
- 2022-11-30 CN CN202211544547.9A patent/CN115866277A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116800976A (en) * | 2023-07-17 | 2023-09-22 | 武汉星巡智能科技有限公司 | Audio and video compression and restoration method, device and equipment for infant with sleep |
| CN116800976B (en) * | 2023-07-17 | 2024-03-12 | 武汉星巡智能科技有限公司 | Audio and video compression and restoration method, device and equipment for infant with sleep |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10950009B2 (en) | AI encoding apparatus and operation method of the same, and AI decoding apparatus and operation method of the same | |
| Wang et al. | Objective quality assessment and perceptual compression of screen content images | |
| US8787454B1 (en) | Method and apparatus for data compression using content-based features | |
| US8165204B2 (en) | Resource allocation for frame-based controller | |
| US6011864A (en) | Digital image coding system having self-adjusting selection criteria for selecting a transform function | |
| Zhang et al. | Practical image quality metric applied to image coding | |
| Rehman et al. | SSIM-inspired perceptual video coding for HEVC | |
| EP1680928A2 (en) | Methods of processing digital image and/or video data including luminance filtering based on chrominance data | |
| US20200068200A1 (en) | Methods and apparatuses for encoding and decoding video based on perceptual metric classification | |
| Wang et al. | Perceptual screen content image quality assessment and compression | |
| CN112738515A (en) | Quantization parameter adjustment method and apparatus for adaptive quantization | |
| CN109324778B (en) | Compression method for compensation pressure | |
| CN108521572B (en) | Residual filtering method based on pixel domain JND model | |
| CN115866277A (en) | Video encoding control method, video encoding control apparatus, and readable storage medium | |
| US11494946B2 (en) | Data compression device and compression method configured to gradually adjust a quantization step size to obtain an optimal target quantization step size | |
| JP3105335B2 (en) | Compression / expansion method by orthogonal transform coding of image | |
| Ström et al. | PACKMAN: texture compression for mobile phones | |
| Joshua et al. | Comparison of DCT and DWT image compression | |
| CN110740324B (en) | Coding control method and related device | |
| CN115802038A (en) | Quantization parameter determination method and device and video coding method and device | |
| WO2012118569A1 (en) | Visually optimized quantization | |
| US20060039619A1 (en) | Image compression using a color visual model | |
| Yang et al. | Contourlet-based image quality assessment for synthesised virtual image | |
| US20230141888A1 (en) | Bicriteria Block Splitting Heuristic For Lossy Compression | |
| Zhang et al. | Divisively normalized sparse coding: toward perceptual visual signal representation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |