CN112634268B - Video quality evaluation method and device and electronic equipment - Google Patents

Video quality evaluation method and device and electronic equipment Download PDF

Info

Publication number
CN112634268B
CN112634268B CN202110032546.5A CN202110032546A CN112634268B CN 112634268 B CN112634268 B CN 112634268B CN 202110032546 A CN202110032546 A CN 202110032546A CN 112634268 B CN112634268 B CN 112634268B
Authority
CN
China
Prior art keywords
quality
sample
video
factor
obtaining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110032546.5A
Other languages
Chinese (zh)
Other versions
CN112634268A (en
Inventor
蔡亮
雷玉荣
吕颖轩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Huoyin Technology Co ltd
Original Assignee
Beijing Huoyin Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Huoyin Technology Co ltd filed Critical Beijing Huoyin Technology Co ltd
Priority to CN202110032546.5A priority Critical patent/CN112634268B/en
Publication of CN112634268A publication Critical patent/CN112634268A/en
Application granted granted Critical
Publication of CN112634268B publication Critical patent/CN112634268B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30168Image quality inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Quality & Reliability (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

The application discloses a video quality evaluation method, a video quality evaluation device and electronic equipment, wherein the method comprises the following steps: obtaining a target video to be evaluated; inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model; the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos, and the quality evaluation model is obtained at least based on factor values of the sample videos on at least one preset quality factor, wherein the quality factor is a factor influencing the quality of the sample videos; and obtaining the video quality grade corresponding to the target video according to the output result. Therefore, the quality evaluation model trained by the method can be more accurate, and the video quality grade obtained by the quality evaluation model is more accurate.

Description

Video quality evaluation method and device and electronic equipment
Technical Field
The present disclosure relates to the field of video processing technologies, and in particular, to a video quality evaluation method and apparatus, and an electronic device.
Background
Due to video coding errors, distortion of different degrees than the original video can be caused, so that the video quality is reduced. How to evaluate the quality of video is a central issue of concern for video providers.
In the current scheme for evaluating video quality, a scheme for subjectively evaluating video quality is generally adopted, namely: the average subjective opinion score MOS (Mean Opinion Score) value is used directly by the test person to give the video quality evaluation result.
Therefore, there is a defect that video quality evaluation is inaccurate.
Disclosure of Invention
In view of this, the present application provides a video quality evaluation method, apparatus and electronic device, which are used to solve the technical problem of inaccurate video quality evaluation in the prior art.
The application provides a video quality evaluation method, which comprises the following steps:
obtaining a target video to be evaluated;
inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, and the quality factor is a factor influencing the quality of the sample videos;
And obtaining the video quality grade corresponding to the target video according to the output result.
In the above method, preferably, the output result includes probability values of the target video on a plurality of preset quality levels;
the obtaining, according to the output result, a video quality level corresponding to the target video includes:
and obtaining a target quality grade with the probability value meeting the evaluation condition from the preset quality grades, and taking the target quality grade as the video quality grade corresponding to the target video.
In the above method, preferably, the output result includes a multidimensional vector feature extracted by the quality evaluation model from the target video;
the obtaining, according to the output result, a video quality level corresponding to the target video includes:
performing dimension reduction processing on the multidimensional vector feature to obtain at least one target feature;
classifying the target features to obtain target quality grades corresponding to the target video in a plurality of preset quality grades, and taking the target quality grades as video quality grades corresponding to the target video.
In the above method, preferably, the quality evaluation model is obtained by training in the following manner:
Sequentially taking each sample video as an input sample of the quality evaluation model to obtain a predicted quality grade of the quality evaluation model output aiming at the sample video;
obtaining a loss function value according to the predicted quality grade and the sample quality label;
and under the condition that the preset training ending condition is not met, adjusting the model parameters of the quality evaluation model according to the loss function value until the training ending condition is met.
The method, preferably, the sample quality tag of the sample video is obtained by:
obtaining factor values of the sample video on one or more quality factors;
processing the factor values through a membership function to obtain membership degrees of the target video corresponding to each of a plurality of preset quality ranges on each quality factor;
according to the membership degree, obtaining an initial quality range corresponding to each quality factor of the sample video;
obtaining a sample quality range of the sample video according to the initial quality range of the sample video corresponding to each quality factor;
And obtaining a sample quality grade of the sample video according to the sample quality range of the sample video, wherein the sample quality grade is marked as a sample quality label of the sample video.
In the above method, preferably, obtaining a sample quality range of the sample video according to an initial quality range of the sample video corresponding to each quality factor includes:
and processing the initial quality range corresponding to each quality factor of the sample video according to the factor weight of the quality factor to obtain the sample quality range of the sample video.
In the above method, preferably, the factor weight of the quality factor is obtained by a hierarchical analysis method.
According to the above method, preferably, according to the membership degree, obtaining an initial quality range corresponding to each quality factor of the sample video includes:
and processing the membership degree of the target video corresponding to each preset quality range on each quality factor according to the grade value corresponding to each preset quality range so as to obtain the initial quality range corresponding to the sample video on the quality factor.
The application also provides a video quality evaluation device, which comprises:
The video obtaining unit is used for obtaining a target video to be evaluated;
the model running unit is used for inputting the target video into a pre-trained quality evaluation model so as to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, and the quality factor is a factor influencing the quality of the sample videos;
and the evaluation obtaining unit is used for obtaining the video quality grade corresponding to the target video according to the output result.
The application also provides an electronic device, comprising:
a memory for storing an application program and data generated by the operation of the application program;
a processor for executing the application program to realize:
obtaining a target video to be evaluated;
inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, and the quality factor is a factor influencing the quality of the sample videos;
And obtaining the video quality grade corresponding to the target video according to the output result.
According to the technical scheme, in the video quality evaluation method, the video quality evaluation device and the electronic equipment disclosed by the application, a quality evaluation model is trained by utilizing a plurality of sample videos with sample quality labels in advance, and the sample quality labels are obtained based on factor values of the sample videos on one or more quality factors affecting the quality of the sample videos, so that after the target videos to be evaluated are obtained, the target videos are processed by utilizing the quality evaluation model, and the video quality grade of the target videos is obtained. Therefore, the quality evaluation model trained by the method can be more accurate, and the video quality grade obtained by the quality evaluation model is more accurate relative to the evaluation result given by the MOS value, so that the purpose of improving the quality evaluation accuracy is achieved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a video quality evaluation method according to an embodiment of the present application;
fig. 2-3 are partial flowcharts of a video quality evaluation method according to a first embodiment of the present application;
fig. 4 is a schematic structural diagram of a video quality evaluation apparatus according to a second embodiment of the present application;
fig. 5 to fig. 6 are schematic diagrams of another structure of a video quality evaluation apparatus according to a second embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to a third embodiment of the present application;
fig. 8 to fig. 9 are respectively flowcharts of a process example suitable for quality evaluation of a movie video according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Referring to fig. 1, a flowchart of an implementation of a video quality evaluation method according to an embodiment of the present application is provided, where the method may be applied to an electronic device capable of performing data processing, such as a computer or a server. The technical scheme in the embodiment is mainly used for improving the accuracy of quality evaluation of video.
Specifically, the method in this embodiment may include the following steps:
step 101: and obtaining the target video to be evaluated.
The target video is a video that needs quality evaluation, such as a movie video or a monitoring video.
Specifically, in this embodiment, after receiving the evaluation request, the target video corresponding to the evaluation request may be acquired.
In one implementation manner, in this embodiment, the target video sent by other devices may be received, for example, the server in this embodiment receives a movie video clip that needs to be subjected to quality evaluation and is sent by the client;
in another implementation manner, in this embodiment, a monitoring video segment corresponding to an evaluation request generated by a video playing application may be read in a storage area of a server.
Step 102: and inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model.
The quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, wherein the sample quality labels represent sample quality grades of the sample videos, such as quality labels 1-5 representing quality grades from poor, medium, good to excellent, and the like, and the sample quality labels are at least obtained based on factor values of the sample videos on at least one preset quality factor, wherein the quality factor is a factor affecting the quality of the sample videos, such as resolution, frame rate, code rate (or bit rate) and the like.
Specifically, the quality factors may be preset according to the user requirements, and these quality factors form a factor set. The sample quality tags for each sample video are obtained in this embodiment based on the factor values of each sample video on these quality factors. For example, in this embodiment, after performing a process such as a weighted sum on these factor values, a process such as a level quantization may be performed, thereby obtaining a sample quality tag for each sample video.
It should be noted that, the quality evaluation model in this embodiment may be a machine learning model constructed based on a machine learning algorithm, such as a neural network model constructed based on a convolutional neural network algorithm. The quality evaluation model can comprise an input layer, an hidden layer, an output layer, a convolution layer, a pooling layer, a full-connection layer and other results, each layer comprises a plurality of operation units, and each operation unit of each layer corresponds to a corresponding model parameter. After the quality evaluation model is constructed, performing multiple rounds of iterative training on the quality evaluation model through a plurality of sample videos, so that model parameters in the quality evaluation model are optimized, and the quality evaluation model subjected to model parameter optimization can evaluate the video quality level of the target video.
In the specific implementation, the trained quality evaluation model can perform processing such as feature extraction and feature classification on the target video on the basis of optimized model parameters, so that an output result is obtained.
In one implementation, the output result may include multidimensional vector features extracted from the target video by the quality assessment model.
In another implementation, the output result may further include probability values of the target video obtained after classifying the vector features at a plurality of preset quality levels.
Step 103: and obtaining the video quality grade corresponding to the target video according to the output result.
Specifically, in this embodiment, after the output result is calculated, a video quality level of the target video is obtained, for example, the video quality level corresponding to the target video is 4, which indicates that the video quality level of the target video is good.
In one implementation manner, in the case where the output result includes probability values of the target video on a plurality of preset quality levels, when obtaining the video quality level corresponding to the target video according to the output result in step 103, the following manner may be implemented:
first, a target quality level whose probability value satisfies an evaluation condition is obtained among preset quality levels, and then the target quality level is taken as a video quality level corresponding to a target video.
The preset quality level may be understood as a preset standard quality level, and may be specifically represented by 1-5, which corresponds to quality levels of poor video quality, poor frequency quality, medium frequency quality, good frequency quality, and good frequency quality, respectively. These preset quality levels may be preset according to requirements, but may of course be set to other numbers of quality levels.
And the evaluation condition may be that the probability value is greater than or equal to the probability threshold value or that the probability value is ordered from large to small in the first bit. Based on this, in the present embodiment, a preset quality level having a probability value greater than or equal to the probability threshold or the maximum is obtained as a target quality level, such as level 4, and thus this target quality level is taken as a video quality level corresponding to the target video, such as target quality level 4, corresponding to the video quality level being good.
In another implementation manner, in the case where the output result includes the multidimensional vector feature of the target video extracted by the quality evaluation model, step 103 may be implemented in the following manner when obtaining the video evaluation result corresponding to the target video according to the output result:
firstly, performing dimension reduction processing on multidimensional vector features contained in an output result to obtain at least one target feature, then classifying the target feature to obtain a target quality grade corresponding to a target video in a plurality of preset quality grades, and finally taking the target quality grade as a video quality grade of the target video.
In this embodiment, the multidimensional vector feature in the output result may be reduced in dimension by using a primary coordinate analysis method pcoa (principal co-ordinates analysis), a support vector machine svm (support vector machines) or a neural network, so as to reduce the number of features participating in the subsequent feature classification, thereby reducing the calculation amount and improving the rate.
In addition, in this embodiment, the target feature may be classified by a preset classification algorithm, such as SVM, to obtain a target quality level, such as level 5, corresponding to the target video in a plurality of preset quality levels, where the target quality level is regarded as a video quality level, such as level 5, corresponding to the video quality level being better.
As can be seen from the foregoing, in the video quality evaluation method provided in the first embodiment of the present application, a quality evaluation model is trained in advance by using a plurality of sample videos with sample quality labels, and since the sample quality labels are obtained based on factor values of the sample videos on one or more quality factors affecting the quality of the sample videos, after obtaining a target video to be evaluated, the target video is processed by using the quality evaluation model, so that a quality evaluation result of a video quality level of the target video can be represented. Therefore, in the embodiment, the sample quality label is set by using the factor values of the sample video on a plurality of quality factors so as to be used for training the quality evaluation model, so that the trained quality evaluation model can be more accurate, and the quality evaluation result obtained by using the quality evaluation model is more accurate relative to the evaluation result given by using the MOS value, thereby achieving the purpose of improving the quality evaluation accuracy.
In one implementation, the quality assessment model may train with multiple sample videos as input samples and sample quality labels of the sample videos as output samples, so that the trained quality assessment model can perform quality assessment on the videos. Specifically, the quality assessment model may be trained as follows, as shown in fig. 2:
step 201: and taking the current sample video as an input sample of the quality evaluation model to obtain a predicted quality grade of the quality evaluation model output aiming at the sample video.
The current sample video is a video which is selected from all sample videos and used for training a quality evaluation model. For example, in this embodiment, the current sample video may be selected randomly from sample videos that do not participate in training, or may be selected from sample videos that do not participate in training according to a preset manner.
It should be noted that, the quality evaluation model has initial model parameters, such as the number of neurons and weight parameters in the neural network algorithm, and in this embodiment, after the current sample video is selected, the quality evaluation model with the initial model parameters is used to perform quality evaluation processing on the current sample video to obtain a predicted quality grade, such as 2 or 5.
Step 202: and obtaining a loss function value according to the predicted quality grade and the sample quality label.
In this embodiment, the predicted quality level and the sample quality label may be compared in value, or the predicted quality level and the sample quality label may be input into a loss function to obtain a loss function value, where the loss function value characterizes the difference between the predicted quality level and the sample quality label.
Step 203: whether the training end condition is satisfied is determined, if not, step 204 is performed, otherwise, the training is ended.
Wherein the training end condition may include: the change amount of the loss function value tends to 0 or the loss function value is smaller than or equal to the loss threshold value, and/or the training frequency of the quality evaluation model is larger than or equal to the frequency threshold value. And under the condition that the training ending condition is met, the training of the quality evaluation model is characterized, and if the training ending condition is not met, the quality evaluation model needs to be adjusted and the training needs to be continued.
It should be noted that, the loss threshold value and the frequency threshold value may be preset or modified according to the requirement.
Step 204: and adjusting model parameters of the quality evaluation model according to the loss function value, selecting the next sample video as the current sample video, and returning to the execution step 201 until the training ending condition is met, wherein the loss function value is smaller than or equal to a loss threshold value or the training frequency of the quality evaluation model is larger than or equal to a frequency threshold value.
For example, in this embodiment, the parameter value of the model parameter in the quality evaluation model may be increased or decreased according to the magnitude of the loss function value, so as to pursue that the training ending condition can be satisfied when the next training is performed, for example, the loss function value is reduced below the loss threshold or tends to be unchanged, or the training frequency exceeds the frequency threshold.
In addition, after the training of the quality evaluation model is completed, in this embodiment, the quality evaluation model may also be tested through one or more test videos obtained in advance, so as to determine whether the quality evaluation model meets the quality evaluation condition. Wherein the test video has a test quality label that characterizes a test quality rating of the test video.
For example, taking a test video as an input sample, obtaining a test quality grade of a quality evaluation model output aiming at the test video, obtaining a loss function value according to the test quality grade and a test quality label of the test video, judging whether the loss function value is smaller than or equal to a loss threshold value, if the loss function value is smaller than the loss threshold value, characterizing the quality evaluation model as a condition conforming to quality evaluation, otherwise, continuing to adjust model parameters of the quality evaluation model according to the loss function value until the loss function value is smaller than or equal to the loss threshold value or tends to be unchanged.
Based on the above implementation, the sample quality tags for each sample video can be obtained as follows, as shown in fig. 3:
step 301: factor values of the sample video over one or more quality factors, such as resolution values, code rate values, frame rate values, and the like, are obtained.
Step 302: processing the factor values through membership functions to obtain membership degrees of the target video corresponding to each preset quality range in a plurality of preset quality ranges on each quality factor;
the predicted quality range may be obtained by blurring according to each preset quality level, for example, preset quality level 1 corresponds to a preset quality range of 0-59, preset quality level 2 corresponds to a preset quality range of 60-69, preset quality level 3 corresponds to a preset quality range of 70-79, preset quality level 4 corresponds to a preset quality range of 80-89, and preset quality level 5 corresponds to a preset quality range of 90-100.
Specifically, in this embodiment, the factor value may be subjected to blurring processing, so that the membership degree of the target video to each preset quality range on each quality factor is obtained by processing the factor value through a membership function.
Step 303: and obtaining the initial quality range corresponding to the sample video on each quality factor according to the membership degree.
In this embodiment, the membership degree of the target video corresponding to or being referred to as membership to each preset quality range on each quality factor may be processed according to the grade value corresponding to each preset quality range, so as to obtain the initial quality range corresponding to the sample video on the quality factor.
It should be noted that, the level value may be a representation value of the preset quality level corresponding to each preset quality range, such as 1-5, or may be a middle value or an end value of a range in each preset quality range. Based on this, in this embodiment, these ranking values may be used as weights, and the membership degrees of the target video belonging to each preset quality range on the instruction factor may be weighted and summed, so that the corresponding preset quality range is obtained according to the obtained value, and is used as the initial quality range corresponding to the target video on the quality factor.
For example, for quality factor A, the membership degrees of the target video to the preset quality ranges 0-59, 60-69, 70-79, 80-89, and 90-100, respectively, on quality factor A are in order: 0.1, 0.2 and 0.5, and these membership degrees are assigned to respective corresponding ranking values such as: 30. 65, 75, 85 and 95, respectively, to obtain 81.5, thereby obtaining the preset mass range corresponding to 8.5: 80-89, namely the initial quality range corresponding to the target video on the quality factor A, corresponds to the quality grade.
Step 304: and obtaining the sample quality range of the sample video according to the initial quality range of the sample video corresponding to each quality factor.
In this embodiment, the respective factor weights of each quality factor may be obtained in advance, where the factor weights may represent the attention degree of the quality factor, for example, in this embodiment, the factor weights of the quality factors may be obtained by a hierarchical analysis method, specifically, the hierarchical analysis method may layer the quality factors, then the factor weights are configured for the quality factors of each layer, the quality factors on different layers correspond to different factor weights, and the total value of the factor weights of the quality factors may be 1.
Based on this, in this embodiment, the initial quality range corresponding to each quality factor of the sample video may be processed according to the factor weight of the quality factor, so as to obtain the sample quality range of the sample video. Specifically, in this embodiment, the initial quality range corresponding to each quality factor of the sample video is weighted and summed according to the corresponding factor weight, and then the sample quality range of the sample video is obtained according to the obtained value.
For example, for quality factor a, quality factor B, and quality factor C, each of which corresponds to a factor weight of: 0.2, 0.3, and 0.5, the target video corresponds to an initial quality range on quality factor a, quality factor B, and quality factor C, respectively: 60-69, 70-79 and 90-100, based on which the initial mass ranges in this example are: intermediate values of 60-69, 70-79 and 90-100: 65. 75 and 95 respectively weight and sum 0.2, 0.3 and 0.5 to obtain 83, thereby obtaining a sample quality range 80-89 corresponding to the target video, and the quality grade is good.
Step 305: and obtaining the sample quality grade of the sample video according to the sample quality range of the sample video.
In this embodiment, the sample quality range may be defuzzified, so that a corresponding preset quality level is compared according to the sample quality range, where the level is the sample quality level, and based on this, the sample quality level is marked as a sample quality label of the sample video.
Referring to fig. 4, a schematic structural diagram of a video quality evaluation apparatus according to a second embodiment of the present application may be configured in an electronic device capable of performing data processing, such as a computer or a server. The technical scheme in the embodiment is mainly used for improving the accuracy of quality evaluation of video.
Specifically, the apparatus in this embodiment may include the following units:
a video obtaining unit 401 for obtaining a target video to be evaluated;
a model running unit 402, configured to input the target video into a pre-trained quality evaluation model, so as to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, and the quality factor is a factor influencing the quality of the sample videos;
And the evaluation obtaining unit 403 is configured to obtain a video quality level corresponding to the target video according to the output result.
As can be seen from the above-mentioned scheme, in the video quality evaluation device provided in the second embodiment of the present application, a quality evaluation model is trained in advance by using a plurality of sample videos with sample quality labels, and since the sample quality labels are obtained based on factor values of the sample videos on one or more quality factors affecting the quality of the sample videos, after obtaining a target video to be evaluated, the target video is processed by using the quality evaluation model, so that a quality evaluation result of a video quality level of the target video can be represented. Therefore, in the embodiment, the sample quality label is set by using the factor values of the sample video on a plurality of quality factors so as to be used for training the quality evaluation model, so that the trained quality evaluation model can be more accurate, and the quality evaluation result obtained by using the quality evaluation model is more accurate relative to the evaluation result given by using the MOS value, thereby achieving the purpose of improving the quality evaluation accuracy.
In one implementation, the output result includes probability values of the target video at a plurality of preset quality levels; the evaluation obtaining unit 403 is specifically configured to: and obtaining a target quality grade with the probability value meeting the evaluation condition from the preset quality grades, and taking the target quality grade as the video quality grade corresponding to the target video.
In one implementation, the output result includes multi-dimensional vector features of the target video extracted by the quality assessment model; the evaluation obtaining unit 403 is specifically configured to: performing dimension reduction processing on the multidimensional vector feature to obtain at least one target feature;
classifying the target features to obtain target quality grades corresponding to the target video in a plurality of preset quality grades, and taking the target quality grades as video quality grades corresponding to the target video.
In one implementation, the apparatus in this embodiment further includes the following units, as shown in fig. 5:
model training unit 404 for: sequentially taking each sample video as an input sample of the quality evaluation model to obtain a predicted quality grade of the quality evaluation model output aiming at the sample video; obtaining a loss function value according to the predicted quality grade and the sample quality label; and under the condition that the preset training ending condition is not met, adjusting the model parameters of the quality evaluation model according to the loss function value until the training ending condition is met.
In one implementation, the apparatus in this embodiment further includes the following units, as shown in fig. 6:
A tag obtaining unit 405 for: obtaining factor values of the sample video on one or more quality factors; processing the factor values through a membership function to obtain membership degrees of the target video corresponding to each of a plurality of preset quality ranges on each quality factor; according to the membership degree, obtaining an initial quality range corresponding to each quality factor of the sample video; obtaining a sample quality range of the sample video according to the initial quality range of the sample video corresponding to each quality factor; and obtaining a sample quality grade of the sample video according to the sample quality range of the sample video, wherein the sample quality grade is marked as a sample quality label of the sample video.
Optionally, the tag obtaining unit 405 is specifically configured to, when obtaining the sample quality range of the sample video: and processing the initial quality range corresponding to each quality factor of the sample video according to the factor weight of the quality factor to obtain the sample quality range of the sample video.
Wherein, the factor weight of the quality factor is obtained by a hierarchical analysis method.
Optionally, when obtaining the initial quality range corresponding to each quality factor of the sample video according to the membership degree, the tag obtaining unit 405 is specifically configured to: and processing the membership degree of the target video corresponding to each preset quality range on each quality factor according to the grade value corresponding to each preset quality range so as to obtain the initial quality range corresponding to the sample video on the quality factor.
It should be noted that, the specific implementation of each unit in this embodiment may refer to the corresponding content in the foregoing, which is not described in detail herein.
Referring to fig. 7, a schematic structural diagram of an electronic device according to a third embodiment of the present application may be an electronic device capable of performing data processing, such as a computer or a server. The technical scheme in the embodiment is mainly used for improving the accuracy of quality evaluation of video.
Specifically, the electronic device in this embodiment may include the following structure:
a memory 701 for storing an application program and data generated by the operation of the application program;
a processor 702, configured to execute the application program to implement: obtaining a target video to be evaluated; inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model; the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, and the quality factor is a factor influencing the quality of the sample videos; and obtaining a video evaluation result corresponding to the target video according to the output result, wherein the video evaluation result represents the video quality grade of the target video.
As can be seen from the foregoing, in the electronic device provided in the third embodiment of the present application, a quality evaluation model is trained in advance by using a plurality of sample videos with sample quality labels, and since the sample quality labels are obtained based on factor values of the sample videos on one or more quality factors affecting the quality of the sample videos, after obtaining a target video to be evaluated, the target video is processed by using the quality evaluation model, so that a quality evaluation result of a video quality level of the target video can be represented. Therefore, in the embodiment, the sample quality label is set by using the factor values of the sample video on a plurality of quality factors so as to be used for training the quality evaluation model, so that the trained quality evaluation model can be more accurate, and the quality evaluation result obtained by using the quality evaluation model is more accurate relative to the evaluation result given by using the MOS value, thereby achieving the purpose of improving the quality evaluation accuracy.
The following describes the technical solution in the present application in detail, taking the evaluation of the video quality of a movie as an example:
firstly, in the current subjective video quality evaluation method, a tester is found to directly give out an evaluation result of video quality by using an MOS value, and no evaluation result of what index influences the video quality is analyzed. Aiming at the defects, the method introduces a fuzzy comprehensive judgment and analytic hierarchy process to establish an index system of multi-factor and multi-level indexes, and quantifies the indexes to obtain an evaluation result.
Moreover, in the current non-reference objective evaluation method of video quality, the influence of multiple frames on the video quality is difficult to consider, and in the application, the video quality is evaluated through the organic fusion of a deep neural network and pcoa. Furthermore, the subjective evaluation system and the objective evaluation system without reference are organically fused.
In order to achieve the above purpose, the present application provides the following technical solutions, and the specific method includes the following steps:
s1, marking video quality through a fuzzy comprehensive judgment result to obtain training data, namely a sample video in the previous text;
s2, training the marked video data by using a convolutional neural network, namely training a quality evaluation model in the previous process;
s3, extracting features by a convolutional neural network, reducing dimensions by PCOA (principal coordinate analysis), and evaluating video quality.
Further, the fuzzy comprehensive evaluation is a method for comprehensively evaluating the video quality by applying a fuzzy transformation principle, can solve the fuzzy concepts existing in production and life, and is expressed by a quantitative method, so that an auxiliary mode is provided for improving the objectivity of the qualitative, and the method has good results for evaluating multi-factor and multi-level complex problems. The specific steps of the fuzzy comprehensive judgment of video quality in S1 are as follows, as shown in fig. 8:
S11, determining a factor set of video quality evaluation:
the factor set is a common set composed of elements of all factors affecting an evaluation object, and is denoted by U. U= (U1, U2, …, ui, …, un), n is the number of factors. Where the element ui represents the ith factor affecting the video quality.
S12, giving a comment set for video quality evaluation:
the evaluation set V is a predetermined set of comments on the evaluation results of the study object. The present application classifies the rating scale for video quality into n, for example: v= { V1, V2, V3, V4, V5}, corresponding to video quality { poor, medium, good, excellent }.
S13, determining fuzzy weights of the video quality evaluated factors:
the departure point and the emphasis point of the judge are not identical in subjective judgment, so that the judging results are different. And judging the video quality within a certain reasonable range by using an analytic hierarchy process. Within a given range, each factor is given a weight. The factors are divided into a plurality of layers by using the analytic hierarchy process, and then the weights of the factors in each layer are obtained.
S14, single-factor evaluation results of video quality evaluation:
according to the membership function, the membership degree of each factor belonging to each grade is obtained, then a single factor evaluation vector is obtained through weighting and other modes, and a single factor evaluation result is obtained after reverse gelatinization.
S15, fuzzy comprehensive judgment results of video quality evaluation:
and after the judgment of all the factors is finished, obtaining the comprehensive evaluation vector. And then defuzzifying the comprehensive evaluation vector to obtain a corresponding video quality level. And marking the video according to the obtained video quality level. The training data is obtained, wherein the training data comprises a plurality of sample videos, and each sample video is provided with a corresponding marked sample finger quality label.
Further, the specific training method in S2 is described as follows:
s21, according to the scheme of training videos in the subjective video quality library obtained in the S1, dividing sample videos in training data into n levels according to different distortion degrees, distributing labels such as 0, 1, 2, 3, … and n to the n levels, and taking the data with the labels as the input of a training model, namely a quality evaluation model in the past.
S22, initializing the convolutional neural network weight in the quality evaluation model.
S23, inputting training data into the model to obtain an output prediction label, and comparing the prediction video quality label with the input video quality label to calculate a loss function value.
And S24, continuously iterating and adjusting the weight according to the loss function value until the training is finished.
Further, the specific steps in S3 are as follows, as shown in fig. 9:
and S31, extracting features by using a trained convolutional neural network, namely extracting vector features of the film video to be evaluated by using a quality evaluation model in the prior art.
S32, obtaining m most important features by using principal coordinate analysis (PCOA), namely, reducing dimension of vector features.
And S33, performing video quality evaluation classification on the feature obtained through dimension reduction through a classification algorithm to obtain the quality grade of the film video.
Compared with the prior art, the fuzzy comprehensive evaluation is introduced into the subjective evaluation method of the video quality, so that the video quality can be evaluated by a single factor and can be evaluated by multiple factors, therefore, the expert capability can be fully exerted, the video quality can be analyzed from multiple angles, the video quality can be marked with higher accuracy, the accuracy of quality evaluation is improved, the PCOA dimension reduction method is introduced, the feature quantity is reduced, and the calculated amount is reduced.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (9)

1. A video quality evaluation method, comprising:
obtaining a target video to be evaluated;
inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, wherein the quality factor is a factor affecting the quality of the sample videos, and the factor values comprise at least one of resolution, frame rate and code rate;
the sample quality tag of the sample video is obtained by: obtaining factor values of the sample video on one or more quality factors; processing the factor values through a membership function to obtain membership degrees of the target video corresponding to each of a plurality of preset quality ranges on each quality factor; according to the membership degree, obtaining an initial quality range corresponding to each quality factor of the sample video; obtaining a sample quality range of the sample video according to the initial quality range of the sample video corresponding to each quality factor; obtaining a sample quality grade of the sample video according to the sample quality range of the sample video, wherein the sample quality grade is marked as a sample quality label of the sample video;
And obtaining the video quality grade corresponding to the target video according to the output result.
2. The method of claim 1, wherein the output result comprises probability values of the target video at a plurality of preset quality levels;
the obtaining, according to the output result, a video quality level corresponding to the target video includes:
and obtaining a target quality grade with the probability value meeting the evaluation condition from the preset quality grades, and taking the target quality grade as the video quality grade corresponding to the target video.
3. The method of claim 1, wherein the output results comprise multi-dimensional vector features of the target video extracted by the quality assessment model;
the obtaining, according to the output result, a video quality level corresponding to the target video includes:
performing dimension reduction processing on the multidimensional vector feature to obtain at least one target feature;
classifying the target features to obtain target quality grades corresponding to the target video in a plurality of preset quality grades, and taking the target quality grades as video quality grades corresponding to the target video.
4. A method according to claim 1, 2 or 3, characterized in that the quality assessment model is trained by:
sequentially taking each sample video as an input sample of the quality evaluation model to obtain a predicted quality grade of the quality evaluation model output aiming at the sample video;
obtaining a loss function value according to the predicted quality grade and the sample quality label;
and under the condition that the preset training ending condition is not met, adjusting the model parameters of the quality evaluation model according to the loss function value until the training ending condition is met.
5. The method of claim 1, wherein obtaining a sample quality range for the sample video from an initial quality range for the sample video for each of the quality factors comprises:
and processing the initial quality range corresponding to each quality factor of the sample video according to the factor weight of the quality factor to obtain the sample quality range of the sample video.
6. The method of claim 5, wherein the factor weights of the quality factors are obtained by analytic hierarchy process.
7. The method of claim 1, wherein obtaining an initial quality range for the sample video for each of the quality factors based on the membership degrees comprises:
and processing the membership degree of the target video corresponding to each preset quality range on each quality factor according to the grade value corresponding to each preset quality range so as to obtain the initial quality range corresponding to the sample video on the quality factor.
8. A video quality evaluation apparatus, comprising:
the video obtaining unit is used for obtaining a target video to be evaluated;
the model running unit is used for inputting the target video into a pre-trained quality evaluation model so as to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, wherein the quality factor is a factor affecting the quality of the sample videos, and the factor values comprise at least one of resolution, frame rate and code rate;
The sample quality tag of the sample video is obtained by: obtaining factor values of the sample video on one or more quality factors; processing the factor values through a membership function to obtain membership degrees of the target video corresponding to each of a plurality of preset quality ranges on each quality factor; according to the membership degree, obtaining an initial quality range corresponding to each quality factor of the sample video; obtaining a sample quality range of the sample video according to the initial quality range of the sample video corresponding to each quality factor; obtaining a sample quality grade of the sample video according to the sample quality range of the sample video, wherein the sample quality grade is marked as a sample quality label of the sample video;
and the evaluation obtaining unit is used for obtaining the video quality grade corresponding to the target video according to the output result.
9. An electronic device, comprising:
a memory for storing an application program and data generated by the operation of the application program;
a processor for executing the application program to realize:
Obtaining a target video to be evaluated;
inputting the target video into a pre-trained quality evaluation model to obtain an output result output by the quality evaluation model;
the quality evaluation model is obtained by training a plurality of sample videos with sample quality labels, the sample quality labels represent sample quality grades of the sample videos and are at least obtained based on factor values of the sample videos on at least one preset quality factor, wherein the quality factor is a factor affecting the quality of the sample videos, and the factor values comprise at least one of resolution, frame rate and code rate;
the sample quality tag of the sample video is obtained by: obtaining factor values of the sample video on one or more quality factors; processing the factor values through a membership function to obtain membership degrees of the target video corresponding to each of a plurality of preset quality ranges on each quality factor; according to the membership degree, obtaining an initial quality range corresponding to each quality factor of the sample video; obtaining a sample quality range of the sample video according to the initial quality range of the sample video corresponding to each quality factor; obtaining a sample quality grade of the sample video according to the sample quality range of the sample video, wherein the sample quality grade is marked as a sample quality label of the sample video;
And obtaining the video quality grade corresponding to the target video according to the output result.
CN202110032546.5A 2021-01-11 2021-01-11 Video quality evaluation method and device and electronic equipment Active CN112634268B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110032546.5A CN112634268B (en) 2021-01-11 2021-01-11 Video quality evaluation method and device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110032546.5A CN112634268B (en) 2021-01-11 2021-01-11 Video quality evaluation method and device and electronic equipment

Publications (2)

Publication Number Publication Date
CN112634268A CN112634268A (en) 2021-04-09
CN112634268B true CN112634268B (en) 2024-01-05

Family

ID=75293875

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110032546.5A Active CN112634268B (en) 2021-01-11 2021-01-11 Video quality evaluation method and device and electronic equipment

Country Status (1)

Country Link
CN (1) CN112634268B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101334893A (en) * 2008-08-01 2008-12-31 天津大学 Fused image quality integrated evaluating method based on fuzzy neural network
CN103841412A (en) * 2014-03-18 2014-06-04 北京邮电大学 Network video user experience quality measurement method and device
CN110958467A (en) * 2019-11-21 2020-04-03 清华大学 Video quality prediction method and device and electronic equipment
CN111192258A (en) * 2020-01-02 2020-05-22 广州大学 Image quality evaluation method and device
CN111212279A (en) * 2018-11-21 2020-05-29 华为技术有限公司 Video quality assessment method and device
CN111507426A (en) * 2020-04-30 2020-08-07 中国电子科技集团公司第三十八研究所 No-reference image quality grading evaluation method and device based on visual fusion characteristics
CN111582150A (en) * 2020-05-07 2020-08-25 江苏日颖慧眼智能设备有限公司 Method and device for evaluating face quality and computer storage medium
CN111915595A (en) * 2020-08-06 2020-11-10 北京金山云网络技术有限公司 Image quality evaluation method, and training method and device of image quality evaluation model

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9706111B2 (en) * 2015-07-08 2017-07-11 Santa Clara No-reference image and video quality evaluation

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101334893A (en) * 2008-08-01 2008-12-31 天津大学 Fused image quality integrated evaluating method based on fuzzy neural network
CN103841412A (en) * 2014-03-18 2014-06-04 北京邮电大学 Network video user experience quality measurement method and device
CN111212279A (en) * 2018-11-21 2020-05-29 华为技术有限公司 Video quality assessment method and device
CN110958467A (en) * 2019-11-21 2020-04-03 清华大学 Video quality prediction method and device and electronic equipment
CN111192258A (en) * 2020-01-02 2020-05-22 广州大学 Image quality evaluation method and device
CN111507426A (en) * 2020-04-30 2020-08-07 中国电子科技集团公司第三十八研究所 No-reference image quality grading evaluation method and device based on visual fusion characteristics
CN111582150A (en) * 2020-05-07 2020-08-25 江苏日颖慧眼智能设备有限公司 Method and device for evaluating face quality and computer storage medium
CN111915595A (en) * 2020-08-06 2020-11-10 北京金山云网络技术有限公司 Image quality evaluation method, and training method and device of image quality evaluation model

Also Published As

Publication number Publication date
CN112634268A (en) 2021-04-09

Similar Documents

Publication Publication Date Title
CN109472318B (en) Method and device for selecting features for constructed machine learning model
CN111932269B (en) Equipment information processing method and device
US11481707B2 (en) Risk prediction system and operation method thereof
CN112396211B (en) Data prediction method, device, equipment and computer storage medium
CN109685104B (en) Determination method and device for recognition model
CN115184193B (en) Automatic testing method, device, equipment and storage medium for cable
CN110634060A (en) User credit risk assessment method, system, device and storage medium
CN111340233A (en) Training method and device of machine learning model, and sample processing method and device
CN109255389B (en) Equipment evaluation method, device, equipment and readable storage medium
US8824783B2 (en) Method and apparatus for measuring video quality using at least one semi-supervised learning regressor for mean observer score prediction
Qiao et al. Trace-driven optimization on bitrate adaptation for mobile video streaming
CN114817034A (en) Test quality analysis method and device and electronic equipment
Krishnan et al. Social influence bias in recommender systems: a methodology for learning, analyzing, and mitigating bias in ratings
CN112634268B (en) Video quality evaluation method and device and electronic equipment
CN113822336A (en) Cloud hard disk fault prediction method, device and system and readable storage medium
CN113420165B (en) Training of classification model and classification method and device of multimedia data
CN113515704B (en) Recommendation effect evaluation method, device, system and computer program product
CN114820074A (en) Target user group prediction model construction method based on machine learning
CN109543707B (en) Semi-supervised change-level software defect prediction method based on three decisions
CN114692888A (en) System parameter processing method, device, equipment and storage medium
CN113011748A (en) Recommendation effect evaluation method and device, electronic equipment and readable storage medium
CN115982455B (en) Flow adjustment method and device based on fuzzy breakpoint regression model and electronic equipment
Sasikumar et al. Real-Time Big Data Analytics for Live Streaming Video Quality Assessment Using Deep Learning
CN116975752A (en) User tag prediction method, device, electronic equipment and computer program product
CN112925924A (en) Multimedia file recommendation method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant