CN114401400B - Video quality evaluation method and system based on visual saliency coding effect perception - Google Patents
Video quality evaluation method and system based on visual saliency coding effect perception Download PDFInfo
- Publication number
- CN114401400B CN114401400B CN202210057728.2A CN202210057728A CN114401400B CN 114401400 B CN114401400 B CN 114401400B CN 202210057728 A CN202210057728 A CN 202210057728A CN 114401400 B CN114401400 B CN 114401400B
- Authority
- CN
- China
- Prior art keywords
- video
- saliency
- coding effect
- effect
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000000694 effects Effects 0.000 title claims abstract description 105
- 230000000007 visual effect Effects 0.000 title claims abstract description 31
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000008447 perception Effects 0.000 title claims abstract description 21
- 238000001514 detection method Methods 0.000 claims abstract description 20
- 230000006835 compression Effects 0.000 claims abstract description 14
- 238000007906 compression Methods 0.000 claims abstract description 14
- 238000005516 engineering process Methods 0.000 claims abstract description 14
- 230000009466 transformation Effects 0.000 claims abstract description 10
- 230000002708 enhancing effect Effects 0.000 claims abstract description 4
- 238000013507 mapping Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000001303 quality assessment method Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 8
- 238000005520 cutting process Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 claims description 5
- CIWBSHSKHKDKBQ-JLAZNSOCSA-N Ascorbic acid Chemical compound OC[C@H](O)[C@H]1OC(=O)C(O)=C1O CIWBSHSKHKDKBQ-JLAZNSOCSA-N 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 230000000903 blocking effect Effects 0.000 claims description 2
- 238000004590 computer program Methods 0.000 description 8
- 238000003860 storage Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 208000018375 cerebral sinovenous thrombosis Diseases 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N17/00—Diagnosis, testing or measuring for television systems or their details
- H04N17/04—Diagnosis, testing or measuring for television systems or their details for receivers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Algebra (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a video quality evaluation method and a system based on visual saliency coding effect perception, wherein a visual saliency model is introduced to extract a video saliency map; then, enhancing the contrast of the salient region of the image through an image gray level transformation technology so as to extract the salient region from the salient map more accurately; and finally, measuring the compression effect of the salient region by using the proposed coding effect detection model so as to realize the mapping from the intensity value of the compression effect to the video quality, and constructing a compressed video quality evaluation model. Experimental results demonstrate the superiority of the proposed model in assessing compressed video quality.
Description
Technical Field
The invention belongs to the technical field of image quality evaluation, and particularly relates to a video quality evaluation method and system based on visual saliency coding effect perception.
Background
Video coding techniques greatly reduce storage capacity and transmission bandwidth. However, lossy compression and variable channel transmission inevitably cause various distortions. As a result, compressed video tends to exhibit visually objectionable distortion (i.e., coding effects), which greatly affects the perceived quality of the video. In order to effectively analyze and improve the user experience, it is necessary to accurately evaluate the visual quality of the video. The subjective Video Quality Assessment (VQA) is the most accurate, reliable reflection of human perception, as it is the quality scored by the viewer. At present, the accuracy of the objective quality evaluation method is measured only by taking the result of subjective quality evaluation as a reference. According to the international telecommunication union standard, MOS and DMOS are adopted to express the subjective quality of video. Thus, MOS and DMOS are the most reliable quality indicators for evaluating the objective quality of video. Subjective experiments, however, are tedious, time consuming and expensive. Therefore, it is imperative to establish a reliable and objective VQA index. The existing reference-free video quality assessment (NR-VQA) algorithm is mostly aimed at traditional video. Some algorithms involve transmission distortions caused by channel errors, such as packet loss and frame freezing.
Disclosure of Invention
In order to make up for the blank and the deficiency of the prior art, further improve the performance of NR-VQA and realize the perception of various coding effects, the invention provides a video quality evaluation method and a video quality evaluation system based on the perception of the visual saliency coding effect so as to evaluate the quality of compressed video.
Firstly, introducing a visual saliency model to extract a video saliency map; then, enhancing the contrast of the salient region of the image through an image gray level transformation technology so as to extract the salient region from the salient map more accurately; and finally, measuring the compression effect of the salient region by using the proposed coding effect detection model so as to realize the mapping from the intensity value of the compression effect to the video quality, and constructing a compressed video quality evaluation model. Experimental results demonstrate the superiority of the proposed model in assessing compressed video quality.
The invention adopts the following technical scheme:
A video quality evaluation method based on visual saliency coding effect perception is characterized by comprising the following steps of: considering the space-time distribution of visual saliency of a video, firstly introducing a visual saliency model to extract a video saliency map; then, enhancing the contrast of the salient region of the image through an image gray level transformation technology so as to extract the salient region from the salient map more accurately; and measuring the compression effect of the salient region by using the proposed coding effect detection model so as to realize the mapping from the intensity value of the compression effect to the video quality, and constructing a compressed video quality evaluation model so as to evaluate the video quality.
Further, the method for extracting the video saliency map by introducing the visual saliency model specifically comprises the following steps of:
Step S11: giving an input video { F t}t, predicting the significance of the video by using a significance ACLNot network, and obtaining a significance map;
Step S12: acquiring time characteristics of the video saliency map by using convLSTM modules of a saliency network;
Step S13: the saliency maps for all frames are combined into a video saliency map V s.
Further, in step S12, the output of convLSTM modules is calculated using equation (1):
(1)
Wherein i t、ft、ot represents an input gate, a forget gate and an output gate, respectively, sigma and tanh are a sigmoid activation function and a hyperbolic tangent function, respectively, a' is a convolution operator, Representing the Hadamard product; all inputs X, memory cell C, hidden gate H and gates i, f, C are three-dimensional tensors with the same dimension, W s and b s are adjustable weights and biases that can be learned by back propagation; the dynamic saliency map is obtained by convolving the hidden gate H with a 1 x 1 kernel.
Further, the contrast of the image salient region is enhanced by an image gray level transformation technology, and the compression effect of the salient region is measured by using the proposed coding effect detection model specifically comprises the following steps:
step S21: increasing the contrast ratio of a salient object of a salient image of a video frame to a background by using an image gray level transformation technology;
step S22: obtaining a binarization image corresponding to the video frame significance image by using a binarization threshold operation;
Step S23: accurately extracting a salient region from each frame of a video, cutting the salient region into image blocks with the size of 72 x 72 and grouping the image blocks;
Step S24: using DenseNet-PR network (from prior art document ：Liqun Lin,Shiqi Yu,Liping Zhou,Weiling Chen,Tiesong Zhao,and Zhou Wang.PEA265：Perceptual Assessment of Video Compression Artifacts.IEEE Transactions on Circuits and Systems for Video Technology(T-CSVT),2020,30(11)：3898-3909.) to implement the perception of video coding effect, the coding effect intensity value for each image block is obtained, and the coding effect intensity value for each frame is calculated assuming that the coding effect intensity value for each pixel in each image block is equal to the coding effect intensity value for that image block, as shown in equations (2) and (3):
where I ij is the coding effect intensity value of an image block of size 72 x 72, N pixel is the total number of pixels per frame of the region of saliency, Intensity value representing kth coding effect per frame,/>An intensity value representing a kth coding effect for each video;
step S25, calculating the coding effect intensity value of the video sequence through the coding effect intensity value of each frame.
Further, the video quality evaluation process specifically includes the following steps:
step S31: the intensity values of the four coding effects, namely the blocking effect, the blurring effect, the color-spill effect and the ringing effect, are matched with the MOS or DMOS values to form a complete data set, and the representation is shown in a formula (4):
Wherein MOS m|DMOSm represents the compressed video subjective quality score mos|dmos for the mth video;
Step S32: the dataset was calculated as 80:20 is randomly divided into a training set and a testing set;
Step S33: inputting four coding effect intensity values of a video sequence into a Bagging-based SVR model, and outputting the predicted quality fraction of the video by using the SVR model;
Step S34: step S32 and step S33 are regarded as training a basic learning machine, and repeated 10 times to obtain 10 basic learning machines in total;
Step S35: and calculating PLCC and SRCC correlation coefficients between the predicted quality score of the compressed video and the subjective and true score MOS/DMOS of the video, so as to realize the prediction of the quality of the compressed video, wherein the prediction is shown in a formula (5):
Wherein f (·) is a sum operation, y l (x) is a prediction output of the first learning machine, and L is the number of learning machines, and the total number of the learning machines is 10; omega l represents the weight of the first learning machine.
Further, the weights of learning machines of the highest three PLCCs are set to 1/3, and the other learning machines are set to 0.
A video quality evaluation system based on visual saliency coding effect perception is characterized in that: the system is based on a computer system and comprises a compressed video significance detection module, a compressed video coding effect detection module and a compressed video quality evaluation module;
The compressed video saliency detection module introduces a visual saliency model to extract a video saliency map;
The compressed video coding effect detection module enhances the contrast of the image salient region through an image gray level transformation technology, and measures the compression effect of the salient region by utilizing the proposed coding effect detection model;
The compressed video quality evaluation module is used for evaluating the quality of the compressed video according to the compressed video quality evaluation model.
Further, the operation process of the compressed video saliency detection module is as follows:
Giving an input video { F t}t, predicting the significance of the video by using a significance ACLNet network, and obtaining a significance map; then, the video saliency map is used for acquiring time characteristics by utilizing convLSTM modules of a saliency network; finally, the saliency maps of all frames are combined into a video saliency map V S.
Further, the operation process of the compressed video coding effect detection module is as follows:
Firstly, increasing the contrast ratio of a salient object of a salient image of a video frame to a background by using an image gray level conversion technology; obtaining a binarization image corresponding to the video frame significance image by using a binarization threshold operation; extracting a salient region accurately from each frame of the video, cutting the salient region into image blocks with the size of 72 x 72 and grouping the image blocks; and then realizing the perception of video coding effect by utilizing DenseNet-PR network [1] to obtain the coding effect intensity value of each image block, and assuming that the coding effect intensity value of each pixel in each image block is equal to the coding effect intensity value of the image block, thereby calculating the coding effect intensity value of each frame, and finally calculating the coding effect intensity value of the video sequence through the coding effect intensity value of each frame.
Further, the operation process of the compressed video quality evaluation module is as follows:
Matching the intensity values of the four coding effects with MOS and DMOS values to form a complete data set, and randomly dividing the data set into a training set and a testing set; inputting four coding effect intensity values of the video sequence into a Bagging-based SVR model, and outputting the predicted quality fraction of the video by using the SVR model; and finally, calculating PLCC and SRCC correlation coefficients between the predicted quality score of the compressed video and the subjective and true score MOS/DMOS of the video, thereby realizing the prediction of the quality of the compressed video.
Compared with the prior art, the visual saliency coding effect-based video quality evaluation method is constructed by the invention and the preferred scheme thereof to evaluate the quality of the compressed video, and the performance and objectivity of the visual saliency coding effect-based video quality evaluation method in evaluating the quality of the compressed video are excellent.
Drawings
FIG. 1 is a schematic diagram of an overall workflow of an embodiment of the present invention;
FIG. 2 is a schematic diagram of the overall structure of a model according to an embodiment of the present invention.
Detailed Description
In order to make the features and advantages of the present patent more comprehensible, embodiments accompanied with figures are described in detail below:
As shown in fig. 1 and fig. 2, the video quality evaluation scheme based on visual saliency coding effect perception provided in this embodiment considers the spatial-temporal distribution of visual saliency of video, and proposes a reference-free compressed video quality evaluation model based on visual saliency coding effect perception, which includes the following steps:
s1, detecting the significance of a compressed video;
s2, detecting the coding effect of the compressed video;
And S3, evaluating the quality of the compressed video.
In an embodiment of the present invention, step S1 is specifically implemented as follows:
Step S11, giving an input video { F t}t, and predicting the significance of the video by using a significance ACLNet network to obtain a significance map;
Step S12, the video saliency map is used for obtaining time characteristics by using a convLSTM module, and the output of the convLSTM module is calculated by using a formula (1):
(1)
Wherein i t、ft、ot represents an input gate, a forget gate and an output gate, respectively, sigma and tanh are a sigmoid activation function and a hyperbolic tangent function, respectively, a' is a convolution operator, Representing the hadamard product. All inputs X, memory cell C, hidden gate H and gates i, f, C are three-dimensional tensors of the same dimension, W s and b s are adjustable weights and biases that can be learned by back propagation. Obtaining a dynamic saliency map by convolving hidden gate H with a 1 x 1 kernel;
Step S13, combining the saliency maps of all frames into a video saliency map V s.
In an embodiment of the present invention, step S2 is specifically implemented as follows:
step S21, firstly, increasing the contrast ratio of a salient object of a salient image of a video frame to a background by using an image gray level conversion technology;
s22, obtaining a binarization map corresponding to the video frame significance map by using a binarization threshold operation;
S23, accurately extracting a salient region from a video frame, cutting the salient region into image blocks with the size of 72x72 and grouping the image blocks;
Step S24, the proposed DenseNet-PR network is utilized to realize the perception of video coding effect, the coding effect intensity value of each image block is obtained, and the coding effect intensity value of each pixel in each image block is assumed to be equal to the coding effect intensity value of the image block, so that the coding effect intensity value of each frame is calculated, and the calculation is shown in formulas (2) and (3):
Where I ij is the coding effect intensity value of a 72 x 72 size image block, N pixel is the total number of pixels per frame of the region of saliency, Intensity value representing kth coding effect per frame,/>An intensity value representing a kth coding effect for each video;
step S25, calculating the coding effect intensity value of the video sequence through the coding effect intensity value of each frame.
In an embodiment of the present invention, step S3 is specifically implemented as follows:
Step S31, first, the intensity values of the four coding effects are matched with MOS (Mean Opinion Score, MOS) |dmos (DIFFERENT MEAN opion Score, DMOS) values to form a complete data set, which can be expressed as shown in formula (4):
Wherein MOS m|DMOSm represents the compressed video quality score mos|dmos for the mth video;
step S32, randomly dividing the data set into a training set and a testing set according to the proportion of 80:20;
Step S33, inputting four coding effect intensity values of the video sequence into a Bagging-based SVR model, and outputting the predicted quality fraction of the video by using the SVR model;
step S34, regarding the steps S32 and S33 as training one basic learning machine, repeating 10 times to obtain 10 basic learning machines in total;
Step S35, finally, calculating PLCC and SRCC correlation coefficients between the predicted quality score of the compressed video and the subjective and true score MOS/DMOS of the video, so as to realize the prediction of the quality of the compressed video, wherein the prediction is shown in a formula (5):
Wherein f (·) is a sum operation, y l (x) is a prediction output of the first learning machine, and L is the number of learning machines, and the total number of learning machines is 10. Omega l represents the weight of the first learning machine. Here we set the weights of the learning machines of the top three PLCCs to 1/3 and the other learning machines to 0.
The above program design scheme provided in this embodiment may be stored in a computer readable storage medium in a coded form, implemented in a computer program, and input basic parameter information required for calculation through computer hardware, and output a calculation result.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations of methods, apparatus (means), and computer program products according to embodiments of the invention. It will be understood that each flow of the flowchart, and combinations of flows in the flowchart, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the invention in any way, and any person skilled in the art may make modifications or alterations to the disclosed technical content to the equivalent embodiments. However, any simple modification, equivalent variation and variation of the above embodiments according to the technical substance of the present invention still fall within the protection scope of the technical solution of the present invention.
The present patent is not limited to the above-mentioned best mode, any person can obtain other various video quality evaluating methods and systems based on the perception of video visual saliency coding effect under the teaching of the present patent, and all equivalent changes and modifications made according to the scope of the present patent application shall be covered by the present patent.
Claims (6)
1. A video quality evaluation method based on visual saliency coding effect perception is characterized by comprising the following steps of: considering the space-time distribution of visual saliency of a video, firstly introducing a visual saliency model to extract a video saliency map; then, enhancing the contrast of the salient region of the image through an image gray level transformation technology so as to extract the salient region from the salient map more accurately; measuring the compression effect of the salient region by using the proposed coding effect detection model to realize the mapping from the intensity value of the compression effect to the video quality, and constructing a compressed video quality evaluation model to evaluate the video quality;
the method for extracting the video saliency map by introducing the visual saliency model specifically comprises the following steps of:
Step S11: giving an input video { F t}t, predicting the significance of the video by using a significance ACLNet network, and obtaining a significance map;
Step S12: acquiring time characteristics of the video saliency map by using convLSTM modules of a saliency network;
Step S13: combining saliency maps of all frames into a video saliency map V S;
in step S12, the output of convLSTM modules is calculated using equation (1):
wherein i t、ft、ot represents an input gate, a forget gate and an output gate, respectively, sigma and tanh are a sigmoid activation function and a hyperbolic tangent function, respectively, a' is a convolution operator, Representing the Hadamard product; all inputs X, memory cell C, hidden gate H and gates i, f, C are three-dimensional tensors with the same dimension, W s and b s are adjustable weights and biases that can be learned by back propagation; obtaining a dynamic saliency map by convolving hidden gate H with a 1 x 1 kernel;
The contrast of the image salient region is enhanced by an image gray level transformation technology, and the compression effect of the salient region is measured by using the proposed coding effect detection model, which specifically comprises the following steps:
step S21: increasing the contrast ratio of a salient object of a salient image of a video frame to a background by using an image gray level transformation technology;
step S22: obtaining a binarization image corresponding to the video frame significance image by using a binarization threshold operation;
Step S23: accurately extracting a salient region from each frame of a video, cutting the salient region into image blocks with the size of 72 x 72 and grouping the image blocks;
Step S24: using DenseNet-PR network to realize the perception of video coding effect, obtaining the coding effect intensity value of each image block, and assuming that the coding effect intensity value of each pixel in each image block is equal to the coding effect intensity value of the image block, thus calculating the coding effect intensity value of each frame, and calculating the coding effect intensity value as shown in formulas (2) and (3):
where I ij is the coding effect intensity value of an image block of size 72 x 72, N pixel is the total number of pixels per frame of the region of saliency, Intensity value representing kth coding effect per frame,/>An intensity value representing a kth coding effect for each video;
Step S25, calculating the coding effect intensity value of the video sequence through the coding effect intensity value of each frame;
the video quality evaluation process specifically comprises the following steps:
Step S31: the intensity values of the four coding effects, namely, the blocking effect, the blurring effect, the color overflow effect and the ringing effect are matched with the MOS or DMOS values to form a complete data set, and the representation is shown in a formula (4):
Wherein MOS m|DMOSm represents the compressed video subjective quality score mos|dmos for the mth video;
Step S32: randomly dividing the data set into a training set and a testing set according to the proportion of 80:20;
Step S33: inputting four coding effect intensity values of a video sequence into a Bagging-based SVR model, and outputting the predicted quality fraction of the video by using the SVR model;
Step S34: step S32 and step S33 are regarded as training a basic learning machine, and repeated 10 times to obtain 10 basic learning machines in total;
Step S35: and calculating PLCC and SRCC correlation coefficients between the predicted quality score of the compressed video and the subjective and true score MOS/DMOS of the video, so as to realize the prediction of the quality of the compressed video, wherein the prediction is shown in a formula (5):
Wherein f (·) is a sum operation, y l (x) is a prediction output of the first learning machine, and L is the number of learning machines, and the total number of the learning machines is 10; omega l represents the weight of the first learning machine.
2. The visual saliency-encoding-effect-based perception video quality assessment method according to claim 1, wherein: the weights of the learning machines of the highest three PLCCs are set to 1/3, and the other learning machines are set to 0.
3. A visual saliency coding effect perception based video quality assessment system for performing the visual saliency coding effect perception based video quality assessment method of claim 1, characterized by: the system is based on a computer system and comprises a compressed video significance detection module, a compressed video coding effect detection module and a compressed video quality evaluation module;
The compressed video saliency detection module introduces a visual saliency model to extract a video saliency map;
The compressed video coding effect detection module enhances the contrast of the image salient region through an image gray level transformation technology, and measures the compression effect of the salient region by utilizing the proposed coding effect detection model;
The compressed video quality evaluation module is used for evaluating the quality of the compressed video according to the compressed video quality evaluation model.
4. A visual saliency-encoding-effect-aware-based video quality assessment system according to claim 3, wherein: the operation process of the compressed video saliency detection module is as follows:
Giving an input video { F t}t, predicting the significance of the video by using a significance ACLNet network, and obtaining a significance map; then, the video saliency map is used for acquiring time characteristics by utilizing convLSTM modules of a saliency network; finally, the saliency maps of all frames are combined into a video saliency map V S.
5. A visual saliency-encoding-effect-aware-based video quality assessment system according to claim 3, wherein: the operation process of the compressed video coding effect detection module is as follows:
Firstly, increasing the contrast ratio of a salient object of a salient image of a video frame to a background by using an image gray level conversion technology; obtaining a binarization image corresponding to the video frame significance image by using a binarization threshold operation; extracting a salient region accurately from each frame of the video, cutting the salient region into image blocks with the size of 72 x 72 and grouping the image blocks; and then realizing the perception of video coding effect by utilizing DenseNet-PR network to obtain the coding effect intensity value of each image block, and assuming that the coding effect intensity value of each pixel in each image block is equal to the coding effect intensity value of the image block, thereby calculating the coding effect intensity value of each frame, and finally calculating the coding effect intensity value of the video sequence through the coding effect intensity value of each frame.
6. A visual saliency-encoding-effect-aware-based video quality assessment system according to claim 3, wherein: the operation process of the compressed video quality evaluation module is as follows:
Matching the intensity values of the four coding effects with MOS and DMOS values to form a complete data set, and randomly dividing the data set into a training set and a testing set; inputting four coding effect intensity values of the video sequence into a Bagging-based SVR model, and outputting the predicted quality fraction of the video by using the SVR model; and finally, calculating PLCC and SRCC correlation coefficients between the predicted quality score of the compressed video and the subjective and true score MOS/DMOS of the video, thereby realizing the prediction of the quality of the compressed video.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210057728.2A CN114401400B (en) | 2022-01-19 | 2022-01-19 | Video quality evaluation method and system based on visual saliency coding effect perception |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210057728.2A CN114401400B (en) | 2022-01-19 | 2022-01-19 | Video quality evaluation method and system based on visual saliency coding effect perception |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114401400A CN114401400A (en) | 2022-04-26 |
| CN114401400B true CN114401400B (en) | 2024-06-14 |
Family
ID=81230661
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210057728.2A Active CN114401400B (en) | 2022-01-19 | 2022-01-19 | Video quality evaluation method and system based on visual saliency coding effect perception |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114401400B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107040776A (en) * | 2017-03-29 | 2017-08-11 | 华南理工大学 | A kind of video quality evaluation method based on HDR |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104539962B (en) * | 2015-01-20 | 2017-12-01 | 北京工业大学 | It is a kind of merge visually-perceptible feature can scalable video coding method |
| CN108462872A (en) * | 2018-05-04 | 2018-08-28 | 南京邮电大学 | A kind of gradient similar video method for evaluating quality based on low frequency conspicuousness |
| CN111711816B (en) * | 2020-07-08 | 2022-11-11 | 福州大学 | Video objective quality evaluation method based on observable coding effect intensity |
| CN113327234B (en) * | 2021-05-31 | 2022-11-08 | 广西大学 | Video redirection quality evaluation method based on space-time saliency classification and fusion |
| CN113810555A (en) * | 2021-09-17 | 2021-12-17 | 福建省二建建设集团有限公司 | Video quality evaluation method based on just noticeable difference and blocking effect |
-
2022
- 2022-01-19 CN CN202210057728.2A patent/CN114401400B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107040776A (en) * | 2017-03-29 | 2017-08-11 | 华南理工大学 | A kind of video quality evaluation method based on HDR |
Non-Patent Citations (1)
| Title |
|---|
| 基于改进四元傅里叶变换的显著性检测及其视频编码应用;李富生;李霞;陈宇;;计算机应用研究;20150531(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114401400A (en) | 2022-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108090902B (en) | Non-reference image quality objective evaluation method based on multi-scale generation countermeasure network | |
| CN108428227B (en) | No-reference image quality evaluation method based on full convolution neural network | |
| CN102333233B (en) | Stereo image quality objective evaluation method based on visual perception | |
| CN103996192B (en) | Non-reference image quality evaluation method based on high-quality natural image statistical magnitude model | |
| CN102209257B (en) | Stereo image quality objective evaluation method | |
| Ma et al. | Reduced-reference image quality assessment in reorganized DCT domain | |
| CN102663747B (en) | Stereo image objectivity quality evaluation method based on visual perception | |
| CN107027023A (en) | VoIP based on neutral net is without reference video communication quality method for objectively evaluating | |
| CN102547368B (en) | Objective evaluation method for quality of stereo images | |
| Yue et al. | Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry | |
| CN107481236A (en) | A kind of quality evaluating method of screen picture | |
| CN107959848A (en) | Universal no-reference video quality evaluation algorithms based on Three dimensional convolution neutral net | |
| CN110599468A (en) | No-reference video quality evaluation method and device | |
| CN109859166A (en) | It is a kind of based on multiple row convolutional neural networks without ginseng 3D rendering method for evaluating quality | |
| CN111709914A (en) | Non-reference image quality evaluation method based on HVS characteristics | |
| CN108513132B (en) | Video quality evaluation method and device | |
| CN109257592B (en) | Stereoscopic video quality objective evaluation method based on deep learning | |
| CN112950579A (en) | Image quality evaluation method and device and electronic equipment | |
| CN111641822A (en) | Method for evaluating quality of repositioning stereo image | |
| CN109741315B (en) | Non-reference image quality objective evaluation method based on deep reinforcement learning | |
| CN113222902B (en) | No-reference image quality evaluation method and system | |
| CN105898279B (en) | A kind of objective evaluation method for quality of stereo images | |
| CN102737380A (en) | Stereo image quality objective evaluation method based on gradient structure tensor | |
| CN114401400B (en) | Video quality evaluation method and system based on visual saliency coding effect perception | |
| CN101895787A (en) | Method and system for subjectively evaluating video coding performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |