CN112767310B - Video quality evaluation method, device and equipment - Google Patents
Video quality evaluation method, device and equipment Download PDFInfo
- Publication number
- CN112767310B CN112767310B CN202011624696.7A CN202011624696A CN112767310B CN 112767310 B CN112767310 B CN 112767310B CN 202011624696 A CN202011624696 A CN 202011624696A CN 112767310 B CN112767310 B CN 112767310B
- Authority
- CN
- China
- Prior art keywords
- video
- evaluated
- images
- frame image
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000006243 chemical reaction Methods 0.000 claims abstract description 153
- 238000012545 processing Methods 0.000 claims abstract description 39
- 238000013145 classification model Methods 0.000 claims description 17
- 238000003062 neural network model Methods 0.000 claims description 5
- 238000001303 quality assessment method Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 8
- 238000012854 evaluation process Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241001184547 Agrostis capillaris Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention provides a video quality evaluation method, a video quality evaluation device and video quality evaluation equipment, and relates to the technical field of communication. The method comprises the following steps: determining a reference video corresponding to the video to be evaluated; performing up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video; determining an up-conversion mode of a frame image in the video to be evaluated according to the target images; and obtaining a quality evaluation result of the video to be evaluated according to the up-conversion mode of the frame image in the video to be evaluated. The scheme of the invention solves the problem of inaccurate video quality evaluation in the prior art.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, and a device for evaluating video quality.
Background
At present, a video with higher definition can be obtained through up-conversion of an original video with lower definition, so that an image has artificial stretching marks on a resolution scale to influence the viewing experience.
The existing video quality evaluation mode uses different bit rates and different coding structures to encode and decode the video to be tested, tests the video before and after encoding and decoding and objectively evaluates and analyzes the video quality. However, the video obtained by different up-conversion methods is not the same, and the up-conversion method is not distinguished by the existing quality evaluation method, so that the quality evaluation result is inaccurate.
Disclosure of Invention
The invention aims to provide a video quality evaluation method, a video quality evaluation device and video quality evaluation equipment so as to evaluate the video quality more accurately.
To achieve the above object, an embodiment of the present invention provides a video quality evaluation method, including:
determining a reference video corresponding to the video to be evaluated;
performing up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video;
determining an up-conversion mode of a frame image in the video to be evaluated according to the target images;
and obtaining a quality evaluation result of the video to be evaluated according to the up-conversion mode of the frame image in the video to be evaluated.
Optionally, the determining, according to the multiple target images, an up-conversion manner of a frame image in the video to be evaluated includes:
obtaining quality parameters of target frame images in the video to be evaluated according to the target images;
determining an up-conversion mode of the target frame image according to the quality parameter;
wherein the quality parameters include: peak signal to noise ratio PSNR and structural similarity SSIM.
Optionally, the obtaining, according to the multiple target images, quality parameters of target frame images in the video to be evaluated includes:
calculating PSNR and SSIM between the target frame image in the video to be evaluated and each image in a first group of images of the target images;
taking the maximum PSNR in the calculated PSNR as the PSNR of the target frame image;
taking the maximum SSIM in the calculated SSIM as the SSIM of the target frame image;
the first group of images is a group of images corresponding to the target frame image in the plurality of target images, and each image corresponds to a different preset up-conversion mode.
Optionally, the determining the up-conversion mode of the target frame image according to the quality parameter includes:
taking a preset up-conversion mode corresponding to PSNR and/or SSIM of the target frame image as the up-conversion mode of the target frame image; or,
and obtaining an up-conversion mode of the target frame image through a multi-category classification model and PSNR and SSIM between the target frame image and each image in the first group of images.
Optionally, the multi-category classification model is a constructed neural network model based on PSNR and SSIM of the frame image, and determines an up-conversion mode of the frame image.
Optionally, the obtaining the quality evaluation result of the video to be evaluated according to the up-conversion mode of the frame image in the video to be evaluated includes:
determining a target up-conversion mode with the largest number of corresponding frame images in the video to be evaluated;
obtaining a quality evaluation result of the video to be evaluated according to the type of the up-conversion mode to which the target up-conversion mode belongs; or,
and acquiring the average value of the quality parameters of the frame image corresponding to the target up-conversion mode, and acquiring the quality evaluation result of the video to be evaluated according to the threshold range to which the average value belongs.
Optionally, the quality evaluation result includes: and whether the video to be evaluated is a video with corresponding definition.
To achieve the above object, an embodiment of the present invention provides a video quality evaluation apparatus including:
the first processing module is used for determining a reference video corresponding to the video to be evaluated;
the second processing module is used for carrying out up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video;
the third processing module is used for determining an up-conversion mode of a frame image in the video to be evaluated according to the plurality of target images;
and the fourth processing module is used for obtaining the quality evaluation result of the video to be evaluated according to the up-conversion mode of the frame images in the video to be evaluated.
To achieve the above object, an embodiment of the present invention provides a video quality evaluation apparatus including a transceiver, a processor, a memory, and a program or instructions stored on the memory and executable on the processor; the processor, when executing the program or instructions, implements the video quality assessment method as described above.
To achieve the above object, an embodiment of the present invention provides a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the steps in the video quality evaluation method as described above.
The technical scheme of the invention has the following beneficial effects:
the method of the embodiment of the invention is to firstly determine a reference video corresponding to the video to be evaluated; then, up-converting the determined reference video by a plurality of preset up-converting modes to obtain a plurality of target images; then, according to the obtained multiple target images, further determining an up-conversion mode of the frame images in the video to be evaluated; finally, combining an up-conversion mode of the frame images in the video to be evaluated to obtain a quality evaluation result of the video to be evaluated. Because the up-conversion mode of the video is considered in the evaluation process, the accuracy of the quality evaluation result is improved.
Drawings
FIG. 1 is a flowchart of a video quality evaluation method according to an embodiment of the present invention;
fig. 2 is a block diagram of a video quality evaluation apparatus according to an embodiment of the present invention;
fig. 3 is a block diagram of a video quality evaluation apparatus according to an embodiment of the present invention;
fig. 4 is a block diagram of a video quality evaluation apparatus according to another embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
In various embodiments of the present invention, it should be understood that the sequence numbers of the following processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
In addition, the terms "system" and "network" are often used interchangeably herein.
In the examples provided herein, it should be understood that "B corresponding to a" means that B is associated with a from which B may be determined. It should also be understood that determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information.
As shown in fig. 1, a video quality evaluation method according to an embodiment of the present invention includes:
step 101, determining a reference video corresponding to a video to be evaluated;
102, performing up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video;
step 103, determining an up-conversion mode of a frame image in the video to be evaluated according to the plurality of target images;
and 104, obtaining a quality evaluation result of the video to be evaluated according to an up-conversion mode of the frame image in the video to be evaluated.
According to the method provided by the embodiment of the invention, according to the steps, aiming at the video to be evaluated, a reference video corresponding to the video to be evaluated is determined; then, up-converting the determined reference video by a plurality of preset up-converting modes to obtain a plurality of target images; then, according to the obtained multiple target images, further determining an up-conversion mode of the frame images in the video to be evaluated; finally, combining an up-conversion mode of the frame images in the video to be evaluated to obtain a quality evaluation result of the video to be evaluated. Because the up-conversion mode of the video is considered in the evaluation process, the accuracy of the quality evaluation result is improved.
For example, for quality evaluation of high-definition HP video, according to the method of the embodiment of the invention, the determination that the HP video is a true/false 4K video can be made by considering the up-conversion mode of the video in the evaluation process.
Optionally, in this embodiment, the reference video is a source video of the video to be evaluated, or a non-source video with specifically the same content as the video to be evaluated.
Here, the source video is a base video for producing a video to be evaluated, for example, for HP video as a video to be evaluated, the source video thereof is a low-definition LP video. And the non-source video with the same content as the video to be evaluated is irrelevant to the production of the video to be evaluated, for example, for the HP video as the video to be evaluated, the non-source video with the same content as the HP video is a land mark high definition video.
It should be appreciated that the mapping relationship between the video to be evaluated and the reference video is preset, and the reference video corresponding to the video to be evaluated can be determined through the mapping relationship. Considering that the source video is used for quality evaluation of the video with high accuracy Yu Feiyuan of the result, the step 101 specifically includes: judging whether the evaluation video has a corresponding source video, if so, taking the source video of the evaluation video as a reference video; and if not, taking the non-source video with the specific content the same as the video to be evaluated as the reference video.
In this embodiment, for the determined reference video, a plurality of target images of the reference video will be obtained by executing step 102, for use in subsequent determination of the up-conversion manner of the frame images in the video to be evaluated. Here, the plurality of preset up-conversion modes used for up-conversion processing of the reference video may be a plurality of interpolation up-conversion modes (such as nearest interpolation, bilinear interpolation, bicubic interpolation, regional interpolation, etc.), or a plurality of neural network super-divisions (such as waipu 2x, meta-Upscale based on the super-resolution convolutional neural network srnn, enhanced super-resolution generation countermeasure network ESRGAN based on the residual dense block RRDB, super-resolution natural enhancement library real-enhancement, etc.). Of course, the preset up-conversion method is not limited to the above, and is not listed here.
And generating a plurality of pictures by performing up-conversion processing on the frame images of the reference video by using the plurality of preset up-conversion modes for each frame image of the reference video, i.e. each picture corresponds to one preset up-conversion mode. Specifically, for a reference video, such as an LP video of an HP video to be evaluated (i.e., a source video of the HP video to be evaluated), after performing up-conversion processing on one frame image (may also be referred to as an LP frame image) in the LP video, in the case that the preset up-conversion mode used is B, B HP frame images are generated: HP (high pressure) 1 ，HP 2 ，…，HP B . In this way, after up-conversion processing is performed on N frame images included in the LP video, the obtained target image is n×b HP frame images. Here, the HP frame image obtained by the LP frame image up-conversion processing is a high-definition image of the same pixel size as the HP video frame image to be evaluated.
After determining the plurality of target images of the reference video, step 103 may be performed. Optionally, in this embodiment, step 103 includes:
obtaining quality parameters of target frame images in the video to be evaluated according to the target images;
determining an up-conversion mode of the target frame image according to the quality parameter;
wherein the quality parameters include: peak signal to noise ratio PSNR and structural similarity SSIM.
Here, in order to determine the up-conversion mode of the video frame image to be evaluated, each frame image of the video to be evaluated is taken as a target frame image, and the up-conversion mode of the target frame image is further determined by obtaining PSNR and SSIM of the target frame image.
Optionally, the obtaining, according to the multiple target images, quality parameters of target frame images in the video to be evaluated includes:
calculating PSNR and SSIM between the target frame image in the video to be evaluated and each image in a first group of images of the target images;
taking the maximum PSNR in the calculated PSNR as the PSNR of the target frame image;
taking the maximum SSIM in the calculated SSIM as the SSIM of the target frame image;
the first group of images is a group of images corresponding to the target frame image in the plurality of target images, and each image corresponds to a different preset up-conversion mode.
Here, B images obtained by performing up-conversion processing on the target frame image, which is the first group of images, are the number of preset up-conversion modes. In this way, after calculating the PSNR and SSIM of the target frame image and the first group of images, the obtained maximum PSNR can be used as the PSNR of the target frame image, and the obtained maximum SSIM can be used as the SSIM of the target frame image.
Wherein PSNR between the target frame image and the current image (i.e. one image in the first group of images) can be calculated by the PSNR calculation formulaObtained. Here, L is the maximum pixel value of the pixel point, such as 255; MSE is mean square error, in particular, +.>m is the image widthThe number of pixels in the degree, n is the number of pixels in the image length, i is the number of pixels in the image width, j is the number of pixels in the image length, X (i, j) represents the pixel value of a pixel in the target frame image (i.e. the number i in the image width and the number j in the image length), and Y (i, j) represents the pixel value of a pixel in the current image in the first group image (i.e. the number i in the image width and the number j in the image length). Therefore, after the PSNR of each image in the target frame image and the first image is calculated by the PSNR calculation formula, the maximum PSNR is the PSNR of the target frame image.
Wherein, the SSIM between the target frame image and the current image (i.e. an image in the first group of images) can be obtained by the structural similarity function SSIM (x, y) between the target frame image and the current image, SSIM (x, y) = [ l (x, y) α ·c(x,y) β ·s(x,y) γ ]. Here, x is a target frame image, y is a current image, l (x, y) is a luminance function of the target frame image and the current image, c (x, y) is a contrast function of the target frame image and the current image, s (x, y) is a structural function of the target frame image and the current image, α is a luminance coefficient, β is a contrast coefficient, and γ is a structural coefficient.
WhileWherein mu x Is the average value mu of pixel values of pixel points on the target frame image y Is the average value sigma of pixel values of pixel points on the current image x 2 For the variance, sigma of pixel values of pixel points on the target frame image y 2 For the variance, sigma, of pixel values of pixel points on the current image xy C is the covariance of the pixel value of the pixel point on the target frame image and the pixel value of the pixel point on the current image 1 ＝(k 1 L) 2 ，c 2 ＝(k 2 L) 2 ，c 3 ＝c 2 /2，k 1 And k 2 Is constant, e.g. k 1 ＝0.01，k 2 ＝0.03。
Alternatively, alpha, beta and gamma are all 1,
of course, when calculating the SSIM, in order to simplify the processing, a window with a preset size may be taken from the image, that is, x is an image corresponding to a window in the target frame image, y is an image corresponding to a window in the current image, then the calculation is performed through a sliding window, and finally, the SSIM average value of each window is taken as the global SSIM.
And (3) performing PSNR and SSIM calculation by selecting each frame image in the video to be evaluated as a target frame image, and finally obtaining the PSNR and SSIM of each frame image of the video to be evaluated.
Optionally, in this embodiment, the determining, according to the quality parameter, an up-conversion manner of the target frame image includes:
taking a preset up-conversion mode corresponding to PSNR and/or SSIM of the target frame image as the up-conversion mode of the target frame image; or,
and obtaining an up-conversion mode of the target frame image through a multi-category classification model and PSNR and SSIM between the target frame image and each image in the first group of images.
That is, for a source video in which the reference video is a video to be evaluated, a preset up-conversion mode corresponding to the PSNR and/or SSIM of the target frame image is used as the up-conversion mode of the target frame image. Of course, for a reference video which is a non-source video with the specific content the same as that of the video to be evaluated, although the up-conversion mode of the target frame image can be determined by adopting the previous mode, the accuracy is poor, and therefore, the up-conversion mode of the target frame image can be determined more accurately by a multi-category classification model. The input of the multi-class classification model is PSNR and SSIM between the target frame image and each image in the first group of images, and the output of the multi-class classification model comprises PSNR and SSIM of the target frame image besides the up-conversion mode of the target frame image.
Optionally, the multi-category classification model is a constructed neural network model based on PSNR and SSIM of the frame image, and determines an up-conversion mode of the frame image.
Specifically, to be evaluatedThe HP video, the reference video is a non-source video with the content being the same as that of the HP video, and PSNR and SSIM between each frame of image of the reference video and each image in the corresponding first group of images are input into the multi-category classification model, and the obtained output is:wherein Z is the number of frame images of the HP video to be evaluated, and label Z Up-conversion for the Z-th frame image, < >>PSNR for the Z-th frame image, and->SSIM, which is the Z-th frame image.
The multi-category classification model is obtained through training of a plurality of sample data, and each group of samples comprises a video to be evaluated, a source video of the video to be evaluated and an up-conversion mode of each frame of image of the video to be evaluated. In the training process, up-conversion processing is carried out on frame images of source videos of videos to be evaluated in the sample by using a plurality of preset up-conversion modes, PSNR and SSIM are calculated by combining corresponding frame images of the videos to be evaluated, the PSNR and SSIM are input into a multi-class classification model, and then an output result is obtained and compared with the up-conversion modes in the sample, and the model is adjusted until training is completed. Wherein the multi-category classification model training sets a loss function ofB is the number of preset up-conversion modes lambda cre To predict the correct coefficients lambda nocre For predicting the wrong coefficients. But->When the predicted correct value is 1, the reverse value is 0; similarly->When the value of the prediction error is 1, the reverse is performedThen 0.
After determining the up-conversion mode of the frame image in the video to be evaluated, step 104 is executed to obtain the quality evaluation result of the video to be evaluated. Optionally, in this embodiment, step 104 includes:
determining a target up-conversion mode with the largest number of corresponding frame images in the video to be evaluated;
obtaining a quality evaluation result of the video to be evaluated according to the type of the up-conversion mode to which the target up-conversion mode belongs; or,
and acquiring the average value of the quality parameters of the frame image corresponding to the target up-conversion mode, and acquiring the quality evaluation result of the video to be evaluated according to the threshold range to which the average value belongs.
In this way, after the up-conversion mode of each frame image in the video to be evaluated is counted, the up-conversion mode with the largest number of corresponding frame images can be determined as the target up-conversion mode, and the target up-conversion mode is used as the up-conversion mode of the video to be evaluated. In this way, on the one hand, according to the type of the up-conversion mode to which the target up-conversion mode belongs, the quality evaluation result of the video to be evaluated can be obtained. Of course, at this time, the type of the up-conversion mode is preset, and the mapping relationship between the quality evaluation result and the up-conversion mode may also be preset. On the other hand, for the target up-conversion mode, the average value of the quality parameters of the corresponding frame image is obtained, and then the quality evaluation result of the video to be evaluated is obtained according to the threshold range of the average value. At this time, the mapping relationship between the quality evaluation result and the different threshold ranges is preset.
For example, in the up-conversion method of the frame images of the video to be evaluated, the number P of frame images corresponding to the up-conversion method a A The duty cycle is the largest among the P frame images of the video to be evaluated,and if the video frequency is greater than or equal to 80%, the up-conversion mode A is the up-conversion mode of the video frequency to be evaluated. Then, P corresponding to the up-conversion mode A in the video to be evaluated A A frame image, the average value of PSNR and SSIM is calculatedThen, the average value is used as an index of quality evaluation, and the first quality evaluation result is obtained by comparing the average value with a threshold range corresponding to the first quality evaluation result; and if the mean value belongs to the threshold range corresponding to the second quality evaluation result, obtaining the second quality evaluation result.
In this embodiment, optionally, the quality evaluation result includes: and whether the video to be evaluated is a video with corresponding definition.
Thus, for the HP video to be evaluated, if the previous example is continued, it corresponds to P of the up-conversion mode A A After calculating the average value of PSNR and SSIM of the frame images, if the average value belongs to a threshold range corresponding to 'true 4K', the HP video to be evaluated can be evaluated as a true 4K video; and conversely, the video is pseudo 4K video.
In summary, according to the method of the embodiment of the present invention, for a video to be evaluated, a reference video corresponding to the video to be evaluated is determined first; then, up-converting the determined reference video by a plurality of preset up-converting modes to obtain a plurality of target images; then, according to the obtained multiple target images, further determining an up-conversion mode of the frame images in the video to be evaluated; finally, combining an up-conversion mode of the frame images in the video to be evaluated to obtain a quality evaluation result of the video to be evaluated. Because the up-conversion mode of the video is considered in the evaluation process, the accuracy of the quality evaluation result is improved.
As shown in fig. 2, a video quality evaluation apparatus according to an embodiment of the present invention includes:
a first processing module 210, configured to determine a reference video corresponding to the video to be evaluated;
the second processing module 220 is configured to perform up-conversion processing on the reference video according to a plurality of preset up-conversion modes, so as to obtain a plurality of target images of the reference video;
a third processing module 230, configured to determine an up-conversion manner of a frame image in the video to be evaluated according to the multiple target images;
and a fourth processing module 240, configured to obtain a quality evaluation result of the video to be evaluated according to an up-conversion manner of the frame image in the video to be evaluated.
Optionally, the third processing module includes:
the first processing sub-module is used for obtaining quality parameters of target frame images in the video to be evaluated according to the plurality of target images;
the second processing sub-module is used for determining an up-conversion mode of the target frame image according to the quality parameter;
wherein the quality parameters include: peak signal to noise ratio PSNR and structural similarity SSIM.
Optionally, the first processing submodule includes:
a calculating unit, configured to calculate PSNR and SSIM between a target frame image in the video to be evaluated and each image in a first group of images of the plurality of target images;
a first processing unit configured to take a maximum PSNR of the calculated PSNRs as a PSNR of the target frame image;
the second processing unit is used for taking the maximum SSIM in the calculated SSIM as the SSIM of the target frame image;
the first group of images is a group of images corresponding to the target frame image in the plurality of target images, and each image corresponds to a different preset up-conversion mode.
Optionally, the second processing sub-module is further configured to:
taking a preset up-conversion mode corresponding to PSNR and/or SSIM of the target frame image as the up-conversion mode of the target frame image; or,
and obtaining an up-conversion mode of the target frame image through a multi-category classification model and PSNR and SSIM between the target frame image and each image in the first group of images.
Optionally, the multi-category classification model is a constructed neural network model based on PSNR and SSIM of the frame image, and determines an up-conversion mode of the frame image.
Optionally, the fourth processing module includes:
the determining submodule is used for determining a target up-conversion mode with the largest number of corresponding frame images in the video to be evaluated;
the third processing sub-module is used for obtaining the quality evaluation result of the video to be evaluated according to the type of the up-conversion mode to which the target up-conversion mode belongs; or,
and acquiring the average value of the quality parameters of the frame image corresponding to the target up-conversion mode, and acquiring the quality evaluation result of the video to be evaluated according to the threshold range to which the average value belongs.
Optionally, the quality evaluation result includes: and whether the video to be evaluated is a video with corresponding definition.
Optionally, the reference video is a source video of the video to be evaluated, or a non-source video with the specific same content as the video to be evaluated.
Aiming at a video to be evaluated, the device determines a reference video corresponding to the video to be evaluated; then, up-converting the determined reference video by a plurality of preset up-converting modes to obtain a plurality of target images; then, according to the obtained multiple target images, further determining an up-conversion mode of the frame images in the video to be evaluated; finally, combining an up-conversion mode of the frame images in the video to be evaluated to obtain a quality evaluation result of the video to be evaluated. Because the up-conversion mode of the video is considered in the evaluation process, the accuracy of the quality evaluation result is improved.
It should be noted that, the device is a device to which the video quality evaluation method is applied, and the implementation manner of the embodiment of the method is applicable to the device, so that the same technical effects can be achieved, and is not described herein again.
As shown in fig. 3, a video quality evaluation apparatus 300 according to an embodiment of the present invention includes a processor 310, where the processor 310 is configured to:
determining a reference video corresponding to the video to be evaluated;
performing up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video;
determining an up-conversion mode of a frame image in the video to be evaluated according to the target images;
and obtaining a quality evaluation result of the video to be evaluated according to the up-conversion mode of the frame image in the video to be evaluated.
Optionally, the processor is further configured to:
obtaining quality parameters of target frame images in the video to be evaluated according to the target images;
determining an up-conversion mode of the target frame image according to the quality parameter;
wherein the quality parameters include: peak signal to noise ratio PSNR and structural similarity SSIM.
Optionally, the processor is further configured to:
calculating PSNR and SSIM between the target frame image in the video to be evaluated and each image in a first group of images of the target images;
taking the maximum PSNR in the calculated PSNR as the PSNR of the target frame image;
taking the maximum SSIM in the calculated SSIM as the SSIM of the target frame image;
the first group of images is a group of images corresponding to the target frame image in the plurality of target images, and each image corresponds to a different preset up-conversion mode.
Optionally, the processor is further configured to:
taking a preset up-conversion mode corresponding to PSNR and/or SSIM of the target frame image as the up-conversion mode of the target frame image; or,
and obtaining an up-conversion mode of the target frame image through a multi-category classification model and PSNR and SSIM between the target frame image and each image in the first group of images.
Optionally, the multi-category classification model is a constructed neural network model based on PSNR and SSIM of the frame image, and determines an up-conversion mode of the frame image.
Optionally, the processor is further configured to:
determining a target up-conversion mode with the largest number of corresponding frame images in the video to be evaluated;
obtaining a quality evaluation result of the video to be evaluated according to the type of the up-conversion mode to which the target up-conversion mode belongs; or,
and acquiring the average value of the quality parameters of the frame image corresponding to the target up-conversion mode, and acquiring the quality evaluation result of the video to be evaluated according to the threshold range to which the average value belongs.
Optionally, the quality evaluation result includes: and whether the video to be evaluated is a video with corresponding definition.
Optionally, the reference video is a source video of the video to be evaluated, or a non-source video with the specific same content as the video to be evaluated.
The video quality evaluation apparatus 300 of the embodiment of the present invention may further include a transceiver 320 for transceiving data under the control of the processor 310.
The video quality evaluation device of this embodiment, for a video to be evaluated, determines a reference video corresponding thereto by first determining the reference video; then, up-converting the determined reference video by a plurality of preset up-converting modes to obtain a plurality of target images; then, according to the obtained multiple target images, further determining an up-conversion mode of the frame images in the video to be evaluated; finally, combining an up-conversion mode of the frame images in the video to be evaluated to obtain a quality evaluation result of the video to be evaluated. Because the up-conversion mode of the video is considered in the evaluation process, the accuracy of the quality evaluation result is improved.
A video quality evaluation apparatus according to another embodiment of the present invention, as shown in fig. 4, includes a transceiver 410, a processor 400, a memory 420, and a program or instructions stored on the memory 420 and executable on the processor 400; the processor 400, when executing the program or instructions, implements the above-described application to video quality assessment methods.
The transceiver 410 is configured to receive and transmit data under the control of the processor 400.
Wherein in fig. 4, a bus architecture may comprise any number of interconnected buses and bridges, and in particular one or more processors represented by processor 400 and various circuits of memory represented by memory 420, linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. The bus interface provides an interface. Transceiver 410 may be a number of elements, i.e., including a transmitter and a receiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 400 is responsible for managing the bus architecture and general processing, and the memory 420 may store data used by the processor 400 in performing operations.
The readable storage medium of the embodiment of the present invention stores a program or an instruction, which when executed by a processor, implements the steps in the video quality evaluation method described above, and can achieve the same technical effects, and is not described herein again for avoiding repetition.
Wherein the processor is a processor in the video quality evaluation apparatus described in the above embodiment. The readable storage medium includes a computer readable storage medium such as a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a magnetic disk or an optical disk.
It is further noted that many of the functional units described in this specification have been referred to as modules, in order to more particularly emphasize their implementation independence.
In an embodiment of the invention, the modules may be implemented in software for execution by various types of processors. An identified module of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different bits which, when joined logically together, comprise the module and achieve the stated purpose for the module.
Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Likewise, operational data may be identified within modules and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices.
Where a module may be implemented in software, taking into account the level of existing hardware technology, a module may be implemented in software, and one skilled in the art may, without regard to cost, build corresponding hardware circuitry, including conventional Very Large Scale Integration (VLSI) circuits or gate arrays, and existing semiconductors such as logic chips, transistors, or other discrete components, to achieve the corresponding functions. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
The exemplary embodiments described above are described with reference to the drawings, many different forms and embodiments are possible without departing from the spirit and teachings of the present invention, and therefore, the present invention should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will convey the scope of the invention to those skilled in the art. In the drawings, the size of the elements and relative sizes may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Unless otherwise indicated, a range of values includes the upper and lower limits of the range and any subranges therebetween.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that various modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the present invention.
Claims (9)
1. A video quality evaluation method, comprising:
determining a reference video corresponding to the video to be evaluated;
performing up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video;
determining an up-conversion mode of a frame image in the video to be evaluated according to the target images;
obtaining a quality evaluation result of the video to be evaluated according to an up-conversion mode of the frame image in the video to be evaluated;
the determining, according to the multiple target images, an up-conversion mode of a frame image in the video to be evaluated includes:
obtaining quality parameters of target frame images in the video to be evaluated by calculating peak signal-to-noise ratio PSNR and structural similarity SSIM between the target frame images in the video to be evaluated and each image in a first group of images of the plurality of target images;
determining an up-conversion mode of the target frame image according to the quality parameter;
wherein the quality parameters include: peak signal to noise ratio PSNR and structural similarity SSIM.
2. The method according to claim 1, wherein obtaining quality parameters of target frame images in the video to be evaluated according to the plurality of target images comprises:
taking the maximum PSNR in the calculated PSNR as the PSNR of the target frame image;
taking the maximum SSIM in the calculated SSIM as the SSIM of the target frame image;
the first group of images is a group of images corresponding to the target frame image in the plurality of target images, and each image corresponds to a different preset up-conversion mode.
3. The method according to claim 2, wherein determining the up-conversion mode of the target frame image according to the quality parameter comprises:
taking a preset up-conversion mode corresponding to PSNR and/or SSIM of the target frame image as the up-conversion mode of the target frame image; or,
and obtaining an up-conversion mode of the target frame image through a multi-category classification model and PSNR and SSIM between the target frame image and each image in the first group of images.
4. A method according to claim 3, wherein the multi-class classification model is a neural network model that has been constructed based on PSNR and SSIM of the frame image, determining the up-conversion of the frame image.
5. The method according to claim 1, wherein the obtaining the quality evaluation result of the video to be evaluated according to the up-conversion manner of the frame image in the video to be evaluated includes:
determining a target up-conversion mode with the largest number of corresponding frame images in the video to be evaluated;
obtaining a quality evaluation result of the video to be evaluated according to the type of the up-conversion mode to which the target up-conversion mode belongs; or,
and acquiring the average value of the quality parameters of the frame image corresponding to the target up-conversion mode, and acquiring the quality evaluation result of the video to be evaluated according to the threshold range to which the average value belongs.
6. The method of claim 1, wherein the quality assessment results comprise: and whether the video to be evaluated is a video with corresponding definition.
7. A video quality evaluation apparatus, comprising:
the first processing module is used for determining a reference video corresponding to the video to be evaluated;
the second processing module is used for carrying out up-conversion processing on the reference video according to a plurality of preset up-conversion modes to obtain a plurality of target images of the reference video;
the third processing module is used for determining an up-conversion mode of a frame image in the video to be evaluated according to the plurality of target images;
the fourth processing module is used for obtaining a quality evaluation result of the video to be evaluated according to an up-conversion mode of the frame images in the video to be evaluated;
the third processing module is further configured to:
obtaining quality parameters of target frame images in the video to be evaluated by calculating peak signal-to-noise ratio PSNR and structural similarity SSIM between the target frame images in the video to be evaluated and each image in a first group of images of the plurality of target images; determining an up-conversion mode of the target frame image according to the quality parameter; wherein the quality parameters include: PSNR and SSIM.
8. A video quality evaluation apparatus comprising: a transceiver, a processor, a memory, and a program or instructions stored on the memory and executable on the processor; a video quality assessment method according to any one of claims 1 to 6, characterized in that said processor when executing said program or instructions.
9. A readable storage medium having stored thereon a program or instructions which when executed by a processor performs the steps in the video quality assessment method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011624696.7A CN112767310B (en) | 2020-12-31 | 2020-12-31 | Video quality evaluation method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011624696.7A CN112767310B (en) | 2020-12-31 | 2020-12-31 | Video quality evaluation method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112767310A CN112767310A (en) | 2021-05-07 |
| CN112767310B true CN112767310B (en) | 2024-03-22 |
Family
ID=75698921
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011624696.7A Active CN112767310B (en) | 2020-12-31 | 2020-12-31 | Video quality evaluation method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112767310B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101156451A (en) * | 2005-04-12 | 2008-04-02 | 皇家飞利浦电子股份有限公司 | Video processing with region-based multiple-pass motion estimation and update of temporal motion vector candidates |
| CN101156450A (en) * | 2005-04-12 | 2008-04-02 | 皇家飞利浦电子股份有限公司 | Region- based 3drs motion estimation using dynamic asoect ratio of region |
| JP2011109176A (en) * | 2009-11-12 | 2011-06-02 | Nippon Telegr & Teleph Corp <Ntt> | Device and method for multiplexing video, and program |
| CN102982508A (en) * | 2011-06-17 | 2013-03-20 | 索尼公司 | Image processing apparatus and method, program, and recording medium |
| CN103414915A (en) * | 2013-08-22 | 2013-11-27 | 合一网络技术(北京)有限公司 | Quality evaluation method and device for uploaded videos of websites |
| EP2736261A1 (en) * | 2012-11-27 | 2014-05-28 | Alcatel Lucent | Method For Assessing The Quality Of A Video Stream |
| CN103856775A (en) * | 2014-03-18 | 2014-06-11 | 天津大学 | Processing method for subjective evaluation result of stereo video quality |
| CN106210767A (en) * | 2016-08-11 | 2016-12-07 | 上海交通大学 | A kind of video frame rate upconversion method and system of Intelligent lifting fluidity of motion |
| CN109068174A (en) * | 2018-09-12 | 2018-12-21 | 上海交通大学 | Video frame rate upconversion method and system based on cyclic convolution neural network |
| CN109379550A (en) * | 2018-09-12 | 2019-02-22 | 上海交通大学 | Video frame rate upconversion method and system based on convolutional neural networks |
| CN109587474A (en) * | 2018-12-14 | 2019-04-05 | 央视国际网络无锡有限公司 | No-reference video quality evaluating method and device based on distortion restoring degree |
| WO2020080698A1 (en) * | 2018-10-19 | 2020-04-23 | 삼성전자 주식회사 | Method and device for evaluating subjective quality of video |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8848061B2 (en) * | 2012-06-27 | 2014-09-30 | Apple Inc. | Image and video quality assessment |
-
2020
- 2020-12-31 CN CN202011624696.7A patent/CN112767310B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101156451A (en) * | 2005-04-12 | 2008-04-02 | 皇家飞利浦电子股份有限公司 | Video processing with region-based multiple-pass motion estimation and update of temporal motion vector candidates |
| CN101156450A (en) * | 2005-04-12 | 2008-04-02 | 皇家飞利浦电子股份有限公司 | Region- based 3drs motion estimation using dynamic asoect ratio of region |
| JP2011109176A (en) * | 2009-11-12 | 2011-06-02 | Nippon Telegr & Teleph Corp <Ntt> | Device and method for multiplexing video, and program |
| CN102982508A (en) * | 2011-06-17 | 2013-03-20 | 索尼公司 | Image processing apparatus and method, program, and recording medium |
| EP2736261A1 (en) * | 2012-11-27 | 2014-05-28 | Alcatel Lucent | Method For Assessing The Quality Of A Video Stream |
| CN103414915A (en) * | 2013-08-22 | 2013-11-27 | 合一网络技术(北京)有限公司 | Quality evaluation method and device for uploaded videos of websites |
| CN103856775A (en) * | 2014-03-18 | 2014-06-11 | 天津大学 | Processing method for subjective evaluation result of stereo video quality |
| CN106210767A (en) * | 2016-08-11 | 2016-12-07 | 上海交通大学 | A kind of video frame rate upconversion method and system of Intelligent lifting fluidity of motion |
| CN109068174A (en) * | 2018-09-12 | 2018-12-21 | 上海交通大学 | Video frame rate upconversion method and system based on cyclic convolution neural network |
| CN109379550A (en) * | 2018-09-12 | 2019-02-22 | 上海交通大学 | Video frame rate upconversion method and system based on convolutional neural networks |
| WO2020080698A1 (en) * | 2018-10-19 | 2020-04-23 | 삼성전자 주식회사 | Method and device for evaluating subjective quality of video |
| CN109587474A (en) * | 2018-12-14 | 2019-04-05 | 央视国际网络无锡有限公司 | No-reference video quality evaluating method and device based on distortion restoring degree |
Non-Patent Citations (1)
| Title |
|---|
| "面向 HDTV 高刷新率的视频帧速率变化算 法研究";陈学伟等;《新型工业化》;第第1卷卷(第第7期期);第58-70页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112767310A (en) | 2021-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR960012931B1 (en) | Channel error concealing method for classified vector quantized video | |
| Sheikh et al. | A statistical evaluation of recent full reference image quality assessment algorithms | |
| US8804815B2 (en) | Support vector regression based video quality prediction | |
| CN111193923A (en) | Video quality evaluation method and device, electronic equipment and computer storage medium | |
| CN105850129A (en) | Method and device for tone-mapping a high dynamic range image | |
| US11259029B2 (en) | Method, device, apparatus for predicting video coding complexity and storage medium | |
| CN112399176B (en) | Video coding method and device, computer equipment and storage medium | |
| Attar et al. | Image quality assessment using edge based features | |
| JP2005064679A (en) | Image feature value extracting method and image quality evaluating method | |
| Chen et al. | Pixel-level texture segmentation based AV1 video compression | |
| Bohr et al. | A no reference image blur detection using cumulative probability blur detection (cpbd) metric | |
| CN112767310B (en) | Video quality evaluation method, device and equipment | |
| Vora et al. | Analysis of compressed image quality assessments, m | |
| CN113452996A (en) | Video coding and decoding method and device | |
| Ghosh et al. | MO-QoE: Video QoE using multi-feature fusion based optimized learning models | |
| CN110070541B (en) | Image quality evaluation method suitable for small sample data | |
| CN112399177A (en) | Video coding method and device, computer equipment and storage medium | |
| CN115550658B (en) | Data transmission method based on intelligent campus management platform | |
| CN112422956B (en) | Data testing system and method | |
| Kalatehjari et al. | A new reduced-reference image quality assessment based on the SVD signal projection | |
| Lin et al. | EVQA: An ensemble-learning-based video quality assessment index | |
| CN111612766B (en) | Image quality evaluation method and device and electronic equipment | |
| US20110110424A1 (en) | Video Encoder and Data Processing Method | |
| CN113038129A (en) | Method and equipment for acquiring data samples for machine learning | |
| Frants et al. | Blind visual quality assessment for smart cloud-based video storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |