CN115965889A - Video quality assessment data processing method, device and equipment - Google Patents
Video quality assessment data processing method, device and equipment Download PDFInfo
- Publication number
- CN115965889A CN115965889A CN202211483351.3A CN202211483351A CN115965889A CN 115965889 A CN115965889 A CN 115965889A CN 202211483351 A CN202211483351 A CN 202211483351A CN 115965889 A CN115965889 A CN 115965889A
- Authority
- CN
- China
- Prior art keywords
- video
- image
- feature vector
- quality
- key frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The invention provides a video quality assessment data processing method, a device and equipment, wherein the method comprises the following steps: extracting video feature vectors of a target video and key frames in the target video; calculating the image fuzziness, the image color richness and the image brightness of the key frame; and determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a no-reference video evaluation algorithm. The invention realizes the non-reference quality evaluation of the monitoring video, can obtain comprehensive and accurate single quality evaluation indexes, and provides comprehensive and accurate reference and theoretical basis for improving the video quality and evaluating the video quality.
Description
Technical Field
The invention relates to the technical field of video image processing, in particular to a video quality assessment data processing method, device and equipment.
Background
Video generally refers to various techniques for capturing, recording, processing, storing, transmitting, and reproducing a series of still images as electrical signals. When the continuous image change exceeds more than 24 frames of pictures per second, human eyes cannot distinguish a single static picture according to the persistence of vision principle; it appears as a smooth continuous visual effect, so that the continuous picture is called a video. With the advancement of science and technology, videos are gradually popularized in work and life of people, such as: film and television works, video monitoring and the like.
The quality of the video is also uneven, and the video with different quality not only affects the experience of people watching the video, but also affects the accuracy of capturing information in the video, and further affects the use of the video. General evaluations for video quality are classified into subjective and objective. The subjective video quality assessment method selects a batch of non-expert testees, allows the testees to continuously watch a series of test sequences in a controlled environment for about 10-30 minutes, then adopts various statistical methods to allow the testees to Score the quality of the video sequences, and finally obtains a video quality average Score (MOS). Objective video quality assessment automatically calculates video quality by using a specific assessment model, and can be generally divided into three categories, namely full reference, partial reference and no reference. The full-reference method and the partial-reference method require high-definition original video and have limited application scenes. The reference-free evaluation method can evaluate the video quality by directly compressing code streams or decoded videos without any original video information, but the existing reference-free video quality evaluation algorithm has low accuracy, can only reflect certain characteristics of the videos and is lack of a comprehensive and accurate interpretable video quality evaluation algorithm.
How to provide a scheme capable of accurately and comprehensively evaluating the video quality is a technical problem which needs to be solved urgently in the field.
Disclosure of Invention
In view of the above, embodiments of the present invention provide a video quality assessment data processing method, apparatus, and device to obviate or mitigate one or more of the disadvantages in the prior art.
One aspect of the present invention provides a video quality assessment data processing method, including the steps of:
extracting video feature vectors of a target video and key frames in the target video;
calculating the image fuzziness, the image color richness and the image brightness of the key frame;
and determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a no-reference video evaluation algorithm.
In some embodiments of the present invention, the extracting the video feature vector of the target video includes:
extracting a frame-level feature vector from the target video by adopting a convolutional neural network algorithm to serve as a space domain feature vector of the target video;
dividing the target video into a plurality of video blocks, and calculating a motion vector of each video block by adopting a search algorithm to serve as a time domain feature vector of the target video;
and taking the spatial domain feature vector and the time domain feature vector as video feature vectors of the target video.
In some embodiments of the present invention, the method for calculating the image blur degree of the key frame includes:
and calculating the image fuzziness of the key frame from different dimensions by adopting various gradient functions.
In some embodiments of the present invention, the determining, by using a no-reference video evaluation algorithm, a video quality score of the target video according to the video feature vector, the image blur, the image color richness, and the image brightness of the key frame includes:
and determining the video quality comprehensive score of the target video and the single feature scores corresponding to different features by adopting a non-reference video evaluation algorithm according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame.
In some embodiments of the present invention, the determining, by using a no-reference video evaluation algorithm, a video quality score of the target video according to the video feature vector, the image blur, the image color richness, and the image brightness of the key frame includes:
acquiring a sample video set, wherein the sample video comprises a plurality of distorted videos and quality labels corresponding to the distorted videos;
extracting sample video feature vectors and sample video key frames of all distorted videos;
calculating the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each distorted video based on the sample key frames;
taking the sample video characteristic vector, the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each distorted video as input, taking the quality label corresponding to each distorted video as output, and training a non-reference video evaluation model;
and inputting the video characteristic vector of the target video, the image fuzziness, the image color richness and the image brightness of the key frame into a trained non-reference video evaluation model, and determining the video quality score of the target video.
Another aspect of the present invention provides a video quality assessment data processing apparatus, comprising:
the data extraction module is used for extracting video feature vectors of a target video and key frames in the target video;
the key frame feature extraction module is used for calculating the image fuzziness, the image color richness and the image brightness of the key frame;
and the video quality evaluation module is used for determining the video quality score of the target video according to the video characteristic vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a non-reference video evaluation algorithm.
In some embodiments of the invention, the data extraction module is specifically configured to:
extracting a frame-level feature vector from the target video by adopting a convolutional neural network algorithm to serve as a space domain feature vector of the target video;
dividing the target video into a plurality of video blocks, and calculating a motion vector of each video block by adopting a search algorithm to serve as a time domain feature vector of the target video;
and taking the spatial domain feature vector and the time domain feature vector as video feature vectors of the target video.
In some embodiments of the present invention, the video quality evaluation module is specifically configured to:
the method for determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting the non-reference video evaluation algorithm comprises the following steps:
and determining the video quality comprehensive score of the target video and the single characteristic score corresponding to different characteristics according to the video characteristic vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a non-reference video evaluation algorithm.
Another aspect of the present invention provides a video quality assessment data processing apparatus comprising a processor and a memory, the memory having stored therein computer instructions for executing the computer instructions stored in the memory, the apparatus implementing the above-described video quality assessment data processing method when the computer instructions are executed by the processor.
Still another aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, which when executed by a processor, implements the above-described video quality assessment data processing method.
The invention provides a video quality assessment data processing method, device and equipment. The method realizes the non-reference quality evaluation of the monitoring video, can obtain comprehensive and accurate single quality evaluation indexes, and provides comprehensive and accurate reference and theoretical basis for improving the video quality and evaluating the video quality.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the specific details set forth above, and that these and other objects that can be achieved with the present invention will be more clearly understood from the detailed description that follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention. For purposes of illustrating and describing some portions of the present invention, corresponding parts may be exaggerated in the drawings, i.e., may be larger relative to other components in an exemplary device actually made according to the present invention. In the drawings:
FIG. 1 is a flow diagram of a method for processing video quality assessment data provided in one embodiment of the present description;
FIG. 2 is a schematic diagram of a non-reference video quality evaluation principle based on spatio-temporal fusion of convolutional neural networks in one embodiment of the present description;
FIG. 3 is a schematic diagram of the feature vectors of quality assessment of non-reference video and the calculated ambiguity, color and brightness features for the key frames in one embodiment of the present disclosure;
FIG. 4 is a block diagram of an embodiment of a video quality assessment data processing apparatus provided herein;
fig. 5 is a block diagram of a hardware configuration of a video quality estimation data processing server in one embodiment of the present specification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the following embodiments and the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and other details not so related to the present invention are omitted.
It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
It is also noted herein that the term "coupled," if not specifically stated, may refer herein to not only a direct connection, but also an indirect connection in which an intermediate is present.
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals denote the same or similar parts, or the same or similar steps.
The video quality evaluation can be understood as quality grading of video data, and is mainly used for evaluating the quality of the video, providing a data basis for watching videos by subsequent people, or providing guidance suggestions for subsequently improving the video quality. The video quality evaluation in the embodiments of the present specification may be used for quality evaluation of arbitrary video data, such as: video data, video surveillance data, and the like. In some embodiments of the present description, the quality of the surveillance video is evaluated, and video surveillance is an important component of a security system, and is widely applied to many occasions due to its intuition, accuracy, timeliness and rich information content. In recent years, with the rapid development of computers, networks, image processing and transmission technologies, video monitoring technologies have been developed.
The terminal image obtained from the video monitoring system can generate quality degradation of different types and degrees after a series of processing procedures such as sampling, compression, transmission and the like, meanwhile, the imaging quality of the terminal image can also be degraded due to the interference of factors such as shooting environment, weather, illumination, lens pollution and the like, and the quality degradation of the terminal image can be collectively called image distortion. The quality of the terminal image directly influences the accuracy of the human eyes for acquiring the visual information, so that the quality evaluation of the terminal image is significant.
The embodiment of the specification provides a video quality assessment data processing method, which combines a no-reference video quality assessment algorithm and feature vectors such as image fuzziness features, color richness features, brightness distribution features and the like aiming at key frames to realize no-reference quality assessment of monitoring videos, can obtain comprehensive and accurate single quality assessment indexes, and provides comprehensive and accurate reference for improving the quality of the monitoring videos and assessing the quality of the monitoring videos.
Fig. 1 is a schematic flow chart of a video quality assessment data processing method provided in an embodiment of this specification, and as shown in fig. 1, in an embodiment of the video quality assessment data processing method provided in this specification, the method may be applied to a computer, a tablet computer, a server, a smart phone, a smart wearable device, and other terminal devices, and the method may include the following steps:
and 102, extracting video feature vectors of a target video and key frames in the target video.
In a specific implementation process, when performing video quality evaluation in an embodiment of the present specification, a video feature vector of a target video may be extracted first, for example: and performing model training on the video sample data obtained by compressing the code stream or decoding by adopting an intelligent algorithm to train a video feature extraction model, and extracting the video feature vector of the target video by utilizing the trained model. Wherein, specific intelligent algorithm can be selected according to actual need, such as: a random forest algorithm, a convolutional neural network algorithm, and the like, which are not specifically limited in the embodiments of the present specification. Such as: a non-reference video quality evaluation model can be trained based on a convolutional neural network algorithm, and a video feature vector of a target video is extracted by using the trained non-reference video quality evaluation model.
Wherein the video feature vector may include: the image processing apparatus may further include at least one of blocking artifacts, block loss, boundary blurring, exposure, contrast, noise, jitter, stagnation, blending, text boxes, left and right black borders, upper and lower black borders, sliding window effects, spatial and temporal losses, black screens, and the like, or may further include other features according to actual needs, and embodiments of the present specification are not particularly limited.
In some embodiments of the present specification, the extracting the video feature vector of the target video includes:
extracting a frame-level feature vector from the target video by adopting a convolutional neural network algorithm to serve as a space domain feature vector of the target video;
dividing the target video into a plurality of video blocks, and calculating a motion vector of each video block by adopting a search algorithm to serve as a time domain feature vector of the target video;
and taking the spatial domain feature vector and the time domain feature vector as video feature vectors of the target video.
In a specific implementation process, a space domain, i.e., a space domain is also called an image space, and a space composed of image elements directly processes the values of the image elements by using length (distance) as an argument in the image space is called space domain processing. The time domain is the time domain, the argument is time, the horizontal axis is time, and the vertical axis is the change of the signal. In the embodiments of the present description, a non-reference video evaluation algorithm may be used to extract a spatial feature vector and a temporal feature vector of a target video, where the spatial feature vector may be understood as a vector capable of representing spatial features of an image, and the temporal feature vector may represent a vector of temporal features of the image. Fig. 2 is a schematic diagram illustrating a principle of non-reference video quality evaluation based on spatial-temporal fusion of a convolutional neural network in an embodiment of the present disclosure, as shown in fig. 2, in some embodiments of the present disclosure, a convolutional neural network-based method is proposed in a spatial domain to learn spatial domain distortion characteristics, such as: a two-dimensional convolutional neural network can be adopted to directly learn the frame-level feature vector of the video from the pixel level as a space-domain feature vector. A group of characteristics based on the similarity of the adjacent frame block structures are designed on the time domain to represent the time domain distortion information of the video, a group of new characteristics based on the similarity of the corresponding block structures of the adjacent frames of the video is defined, the structural similarity of the corresponding small blocks between the adjacent frames can reflect the time domain quality of the video, the video frame can be divided into smaller blocks, and for each small block, a search algorithm is adopted to estimate a motion vector as a time domain characteristic vector.
And taking the space domain feature vector and the time domain feature vector as video feature vectors of the target video, and taking the video feature vectors as final feature representation of the video such as a monitoring video. As shown in fig. 2, the feature vectors in the empty domain and the feature vectors in the time domain in the number of video samples can be extracted in advance, a linear support vector regression model is trained by using the feature vectors, and the trained linear support vector regression model can be used to predict the initial quality score of the video, which is used as an input of the subsequent quality evaluation and lays a data foundation for the subsequent video quality evaluation. Meanwhile, the extracted space domain characteristic vector and time domain characteristic vector are combined with the image characteristics of the subsequent key frame, and the input of comprehensive video quality evaluation can be performed, so that the comprehensive and accurate evaluation of the video quality is realized.
In addition, a key frame extraction is also needed for the target video, and the key frame can be understood as a frame where a key action in the movement or change of the character or object in the video is located. The method for extracting the key frame can be selected according to actual needs, such as: the method includes extracting key frames based on shot boundaries, extracting key frames based on motion analysis, extracting key frames based on image information, and selecting other methods, which is not specifically limited in the embodiments of the present specification. The number of key frames in the target video is determined according to actual situations, and may be one frame or multiple frames, and the embodiments of this specification are not specifically limited.
And 104, calculating the image fuzziness, the image color richness and the image brightness of the key frame.
In a specific implementation process, after the key frame is extracted, for the possible occurrence of blur, brightness and color defects, the image characteristics of the key frame, such as: the image blur is image clarity, image color richness, image brightness, and the like. Of course, other image features in the key frame may also be calculated according to actual needs, and the embodiment of this specification is not particularly limited.
In some embodiments of the present specification, the method for calculating the image blur of the key frame includes:
and calculating the image fuzziness of the key frame from different dimensions by adopting a plurality of gradient functions.
In a specific implementation process, after the key frame is extracted, image definition scores of different dimensions can be given for a judgment basis according to various gradient functions, and the image blurring degree is evaluated according to the scores. Such as: the image blur of the keyframe may be evaluated using a Brenner gradient function, a Tenengrad gradient function, a Laplacian gradient function, and the like.
The Tenengrad gradient function adopts a Sobel operator to respectively extract gradient values in the horizontal direction and the vertical direction, and the definition of an image of the Tenengrad gradient function and a base is defined as follows:
the form of G (x, y) is as follows:
wherein: t is a given edge detection threshold, G x And G y The convolution of Sobel horizontal and vertical direction edge detection operators at pixel point (x, y), respectively, suggests using the following Sobel operator template to detect edges.
The Laplacian gradient function is basically consistent with the Tenengrad gradient function, and a Laplacian operator is used for replacing a Sobel operator, and the operator is defined as follows:
the definition of image star sharpness based on the Laplacian gradient function is therefore as follows:
wherein G (x, y) is the convolution of the Laplacian operator at the pixel point (x, y).
The Brenner gradient function is a simple gradient evaluation function that is a simple calculation of the square of the difference between the gray levels of two adjacent pixels, and is defined as follows:
wherein: g (x, y) represents the gray value of the pixel point (x, y) corresponding to the image f, and D (f) is the image definition calculation result.
The three gradient functions may be used to calculate the image blur degree in the key frame from different dimensions, and certainly, according to actual needs, other gradient functions may also be used to calculate the blur degree of the key frame, which is not specifically limited in the embodiment of this specification. In addition, the image blurriness calculated by the three methods can be integrated to obtain the integrated score of the image blurriness of the key frame.
By adopting various gradient functions to evaluate the image fuzziness of the key frame from different dimensions, the accuracy and the comprehensiveness of the image fuzziness evaluation are improved.
In addition, after the key frame is extracted, the color richness of the image can be calculated according to the pixel color space representation in the image. We derive its image color quality characteristics:
rg＝R-G (6)
the above two equations show a spatial representation of color, where R is red, G is green, and B is blue. In the first equation, rg is the difference between the red and green channels. In the second equation, yb is the general subtraction of the blue channel, representing the sum of the red and green channels. Next, the standard deviation σ of the two values is calculated before the final colorimetry is calculated rgyb And mean value mu rgyb 。
C＝σ rgyb +0.3μ rgyb (10)
In some embodiments of the present specification, the method for calculating the image brightness of the key frame includes:
calculating the mean and variance of the key frame on the gray level image;
determining exposure information of the image corresponding to the key frame according to the calculated mean value and variance;
and determining the image brightness of the key frame based on the exposure information.
In various specific implementations, it is generally detected whether the luminance is abnormal or not, and by calculating the mean and the variance on the gray scale map, the mean is shifted from the mean point (may be assumed to be 128) and the variance is also decreased when the luminance is abnormal. In the embodiment of the present specification, a mean value and a variance of each pixel point in a gray level image corresponding to a key frame may be calculated, an evaluation parameter for image brightness may be given, and a large number of tests may be performed to adjust a threshold for evaluating whether an image is exposed, so that whether the image is over-exposed or under-exposed may be evaluated accordingly, and thus, the image brightness corresponding to the key frame may be determined. The average gray, variance, and standard deviation of the key frames can be selected as the feature vector for evaluating the brightness quality during monitoring.
And 106, determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a no-reference video evaluation algorithm.
In a specific implementation process, fig. 3 is a schematic diagram of a feature vector for evaluating quality of a non-reference video and features of ambiguity, color and brightness calculated for a key frame in an embodiment of the present specification, and as shown in fig. 3, with the method provided in the above embodiment, a video feature vector of a target video and an initial score of video quality can be extracted, and the video feature vector, the initial score and the initial score are input to a SoftMax layer in combination with image ambiguity, image color richness and image brightness corresponding to a key frame of the target video, and a final score is calculated and output, so as to obtain a video quality score of the target video. In the embodiments of the present description, a no-reference video evaluation algorithm may be used to evaluate video quality, and referring to the description of the above embodiments, the no-reference video evaluation algorithm generally does not need any original video information at all, and can evaluate video quality by directly compressing a bitstream or a decoded video, for example: video Quality Indicators evaluation protocol. The video evaluation model can be constructed in advance by using a no-reference video evaluation algorithm, and the calculated related characteristics of the target video are input into the video evaluation model, so that the quality score of the target video is obtained.
In some embodiments of the present specification, the determining, by using a non-reference video evaluation algorithm, a video quality score of the target video according to the video feature vector, the image blur degree of the keyframe, the image color richness, and the image brightness includes:
and determining the video quality comprehensive score of the target video and the single feature scores corresponding to different features by adopting a non-reference video evaluation algorithm according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame.
In a specific implementation process, when a non-reference video evaluation algorithm is adopted to evaluate the quality of a target video, not only can a video quality comprehensive score of the target video and single feature scores corresponding to different features be obtained, such as: and the single feature scores corresponding to the fuzziness, the noise level, the brightness, the color, the blocking effect and the like. Therefore, the overall quality of the video can be determined, and the quality of a certain aspect of the video can be independently detected, so that a user can conveniently optimize any links such as video acquisition, compression, transmission and the like.
In some embodiments of the present specification, the determining, by using a no-reference video evaluation algorithm, a video quality score of the target video according to the video feature vector, the image blur, the image color richness, and the image brightness of the key frame includes:
acquiring a sample video set, wherein the sample video comprises a plurality of distorted videos and quality labels corresponding to the distorted videos;
extracting sample video feature vectors and sample video key frames of all distorted videos;
calculating the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each distorted video based on the sample key frames;
taking the sample video characteristic vector, the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each distorted video as input, taking the quality label corresponding to each distorted video as output, and training a non-reference video evaluation model;
and inputting the video characteristic vector of the target video, the image fuzziness, the image color richness and the image brightness of the key frame into a trained non-reference video evaluation model, and determining the video quality score of the target video.
In a specific implementation process, when video quality evaluation is performed, a non-reference video evaluation algorithm can be adopted in advance to construct a video quality evaluation model, and then the trained model is utilized to evaluate the quality of a target video. Such as: some distorted videos may be captured as sample videos, such as: and taking the compressed code stream or the decoded monitoring video as sample videos, and then manually marking the quality of each sample video, such as: and manually scoring the quality of each sample video to construct a sample video set. And extracting the sample video feature vectors and the sample video key frames of each sample video, and calculating the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each sample video key frame. And taking the sample video characteristic vector, the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each distorted video as the input of the model, taking the quality label corresponding to each distorted video as the output of the model, performing model training, and training a non-reference video evaluation model. And inputting the extracted video characteristic vector of the target video, the image fuzziness, the image color richness and the image brightness of the key frame into a trained non-reference video evaluation model, and determining the video quality score of the target video by using the non-reference video evaluation model.
As shown in fig. 2 and fig. 3, in the embodiment of the present specification, a deep neural network algorithm may be first adopted to extract a video feature vector of a target video, and a deep neural network model is used to perform non-reference monitoring video quality evaluation on the target video, so as to obtain an average subjective score difference of the target video. And then calculating the characteristics such as the fuzzy degree, the color, the brightness and the like in the key frame of the target video, fusing the characteristics extracted from the key frame with the video characteristic vector of the characteristics of the target video and the average subjective score difference, and further carrying out comprehensive scoring on the target video to realize accurate and comprehensive evaluation on the video quality.
The embodiment of the specification provides a video quality assessment data processing method, which adopts an intelligent algorithm to extract video feature vectors of a target video, then calculates the characteristics of ambiguity, color, brightness and the like corresponding to key frames of the target video, integrates the extracted video feature vectors and the characteristics of ambiguity, color, brightness and the like of the key frames, and adopts a no-reference video evaluation algorithm to assess the quality of the target video. The method realizes the non-reference quality evaluation of the monitoring video, can obtain comprehensive and accurate single quality evaluation indexes, and provides comprehensive and accurate reference and theoretical basis for improving the video quality and evaluating the video quality.
In the present specification, each embodiment of the method is described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. Reference is made to the description of the method embodiments in part.
Based on the video quality assessment data processing method, one or more embodiments of the present specification further provide a device for processing video quality assessment data. The apparatus may include apparatus (including distributed systems), software (applications), modules, components, servers, clients, etc. that use the methods described in embodiments of the present specification in conjunction with hardware where necessary to implement the methods. Based on the same innovative conception, embodiments of the present specification provide an apparatus as described in the following embodiments. Since the implementation scheme of the apparatus for solving the problem is similar to that of the method, the specific apparatus implementation in the embodiment of the present specification may refer to the implementation of the foregoing method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware or a combination of software and hardware is also possible and contemplated.
Specifically, fig. 4 is a schematic block diagram of an embodiment of a video quality assessment data processing apparatus provided in this specification, and as shown in fig. 4, the apparatus provided in this specification may include:
a data extraction module 41, configured to extract a video feature vector of a target video and a key frame in the target video;
a key frame feature extraction module 42, configured to calculate an image blur degree, an image color richness, and an image brightness of the key frame;
and the video quality evaluation module 43 is configured to determine a video quality score of the target video according to the video feature vector, the image blur, the image color richness, and the image brightness of the key frame by using a no-reference video evaluation algorithm.
In some embodiments of the present specification, the data extraction module is specifically configured to:
extracting a frame-level feature vector from the target video by adopting a convolutional neural network algorithm to serve as a space domain feature vector of the target video;
dividing the target video into a plurality of video blocks, and calculating a motion vector of each video block by adopting a search algorithm to serve as a time domain characteristic vector of the target video;
and taking the spatial domain feature vector and the time domain feature vector as video feature vectors of the target video.
In some embodiments of the present description, the video quality evaluation module is specifically configured to:
the method for determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting the non-reference video evaluation algorithm comprises the following steps:
and determining the video quality comprehensive score of the target video and the single feature scores corresponding to different features by adopting a non-reference video evaluation algorithm according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame.
The video quality assessment data processing device provided in the embodiment of the present specification adopts an intelligent algorithm to extract a video feature vector of a target video, then calculates characteristics such as ambiguity, color and brightness corresponding to a key frame of the target video, fuses the extracted video feature vector and characteristics such as ambiguity, color and brightness of the key frame, and adopts a no-reference video evaluation algorithm to perform quality assessment on the target video. The method realizes the non-reference quality evaluation of the monitoring video, can obtain comprehensive and accurate single quality evaluation indexes, and provides comprehensive and accurate reference and theoretical basis for improving the video quality and evaluating the video quality.
It should be noted that the above-described apparatus may also include other embodiments according to the description of the method embodiment. The specific implementation manner may refer to the description of the related method embodiment, and details are not described herein.
The method embodiments provided by the embodiments of the present specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking the example of the operation on the server, fig. 5 is a block diagram of the hardware structure of the video quality assessment data processing server in one embodiment of the present specification, and the computer terminal may be the video quality assessment data processing server or the video quality assessment data processing apparatus in the above embodiment. As shown in fig. 5, the server 10 may include one or more (only one shown) processors 100 (the processors 100 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a non-volatile memory 200 for storing data, and a transmission module 300 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 5 is only an illustration and is not intended to limit the structure of the electronic device. For example, the server 10 may also include more or fewer components than shown in FIG. 5, may also include other processing hardware, such as databases or caches, GPUs, or may have a different configuration than shown in FIG. 5, for example.
The non-volatile memory 200 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the video quality assessment data processing method in the embodiment of the present specification, and the processor 100 executes various functional applications and resource data updates by running the software programs and modules stored in the non-volatile memory 200. Non-volatile memory 200 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the non-volatile memory 200 may further include memory located remotely from the processor 100, which may be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission module 300 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission module 300 includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission module 300 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In accordance with the above method, the present invention further provides an apparatus comprising a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being configured to execute the computer instructions stored in the memory, the apparatus implementing the steps of the method when the computer instructions are executed by the processor.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the foregoing steps of the edge computing server deployment method. The computer readable storage medium may be a tangible storage medium such as Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.
Those of ordinary skill in the art will appreciate that the various illustrative components, systems, and methods described in connection with the embodiments disclosed herein may be implemented as hardware, software, or combinations of both. Whether this is done in hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link.
It is to be understood that the invention is not limited to the precise arrangements and instrumentalities shown. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps, after comprehending the spirit of the present invention.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments in the present invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for processing video quality assessment data, the method comprising the steps of:
extracting video feature vectors of a target video and key frames in the target video;
calculating the image fuzziness, the image color richness and the image brightness of the key frame;
and determining the video quality score of the target video according to the video characteristic vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a non-reference video evaluation algorithm.
2. The method according to claim 1, wherein the extracting the video feature vector of the target video comprises:
extracting a frame-level feature vector from the target video by adopting a convolutional neural network algorithm to serve as a space domain feature vector of the target video;
dividing the target video into a plurality of video blocks, and calculating a motion vector of each video block by adopting a search algorithm to serve as a time domain feature vector of the target video;
and taking the spatial domain feature vector and the time domain feature vector as the video feature vector of the target video.
3. The method according to claim 1, wherein the method for calculating the image blur degree of the key frame comprises:
and calculating the image fuzziness of the key frame from different dimensions by adopting a plurality of gradient functions.
4. The method according to claim 1, wherein the determining the video quality score of the target video according to the video feature vector, the image blur, the image color richness and the image brightness of the key frame by using a no-reference video evaluation algorithm comprises:
and determining the video quality comprehensive score of the target video and the single feature scores corresponding to different features by adopting a non-reference video evaluation algorithm according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame.
5. The method according to claim 1, wherein the determining the video quality score of the target video according to the video feature vector, the image blur, the image color richness and the image brightness of the key frame by using a no-reference video evaluation algorithm comprises:
acquiring a sample video set, wherein the sample video comprises a plurality of distorted videos and quality labels corresponding to the distorted videos;
extracting sample video feature vectors and sample video key frames of all distorted videos;
calculating sample video image fuzziness, sample video image color richness and sample video image brightness of each distorted video based on the sample key frames;
taking the sample video characteristic vector, the sample video image fuzziness, the sample video image color richness and the sample video image brightness of each distorted video as input, taking the quality label corresponding to each distorted video as output, and training a non-reference video evaluation model;
and inputting the video characteristic vector of the target video, the image fuzziness, the image color richness and the image brightness of the key frame into a trained non-reference video evaluation model, and determining the video quality score of the target video.
6. A video quality assessment data processing apparatus, the apparatus comprising:
the data extraction module is used for extracting video feature vectors of a target video and key frames in the target video;
the key frame feature extraction module is used for calculating the image fuzziness, the image color richness and the image brightness of the key frame;
and the video quality evaluation module is used for determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a no-reference video evaluation algorithm.
7. The apparatus of claim 6, wherein the data extraction module is specifically configured to:
extracting a frame-level feature vector from the target video by adopting a convolutional neural network algorithm to serve as a space domain feature vector of the target video;
dividing the target video into a plurality of video blocks, and calculating a motion vector of each video block by adopting a search algorithm to serve as a time domain feature vector of the target video;
and taking the spatial domain feature vector and the time domain feature vector as the video feature vector of the target video.
8. The apparatus of claim 7, wherein the video quality assessment module is specifically configured to:
the method for determining the video quality score of the target video according to the video feature vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting the non-reference video evaluation algorithm comprises the following steps:
and determining the video quality comprehensive score of the target video and the single characteristic score corresponding to different characteristics according to the video characteristic vector, the image fuzziness, the image color richness and the image brightness of the key frame by adopting a non-reference video evaluation algorithm.
9. A video quality assessment data processing apparatus comprising a processor and a memory, wherein said memory has stored therein computer instructions for executing computer instructions stored in said memory, which when executed by the processor, the apparatus implements the steps of the method according to any one of claims 1 to 5.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211483351.3A CN115965889A (en) | 2022-11-24 | 2022-11-24 | Video quality assessment data processing method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211483351.3A CN115965889A (en) | 2022-11-24 | 2022-11-24 | Video quality assessment data processing method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115965889A true CN115965889A (en) | 2023-04-14 |
Family
ID=87352044
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211483351.3A Pending CN115965889A (en) | 2022-11-24 | 2022-11-24 | Video quality assessment data processing method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115965889A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116320405A (en) * | 2023-05-17 | 2023-06-23 | 西安畅榜电子科技有限公司 | Security monitoring video compression storage method |
| CN116843683A (en) * | 2023-08-30 | 2023-10-03 | 荣耀终端有限公司 | Equipment imaging definition evaluation method, system and device |
| CN117119231A (en) * | 2023-10-13 | 2023-11-24 | 深圳市知酷信息技术有限公司 | Video charging regulation and control system based on block chain |
| CN118075449A (en) * | 2024-04-18 | 2024-05-24 | 智广海联(天津)大数据技术有限公司 | Method and device for evaluating camera monitoring video quality |
-
2022
- 2022-11-24 CN CN202211483351.3A patent/CN115965889A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116320405A (en) * | 2023-05-17 | 2023-06-23 | 西安畅榜电子科技有限公司 | Security monitoring video compression storage method |
| CN116320405B (en) * | 2023-05-17 | 2023-10-27 | 西安畅榜电子科技有限公司 | Security monitoring video compression storage method |
| CN116843683A (en) * | 2023-08-30 | 2023-10-03 | 荣耀终端有限公司 | Equipment imaging definition evaluation method, system and device |
| CN116843683B (en) * | 2023-08-30 | 2024-03-05 | 荣耀终端有限公司 | Equipment imaging definition evaluation method, system and device |
| CN117119231A (en) * | 2023-10-13 | 2023-11-24 | 深圳市知酷信息技术有限公司 | Video charging regulation and control system based on block chain |
| CN118075449A (en) * | 2024-04-18 | 2024-05-24 | 智广海联(天津)大数据技术有限公司 | Method and device for evaluating camera monitoring video quality |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115965889A (en) | Video quality assessment data processing method, device and equipment | |
| Yang et al. | Perceptual quality assessment of screen content images | |
| CN114584849B (en) | Video quality evaluation method, device, electronic equipment and computer storage medium | |
| Hosu et al. | The Konstanz natural video database (KoNViD-1k) | |
| CN108933935B (en) | Detection method and device of video communication system, storage medium and computer equipment | |
| CN111079740A (en) | Image quality evaluation method, electronic device, and computer-readable storage medium | |
| CN104023230B (en) | A kind of non-reference picture quality appraisement method based on gradient relevance | |
| CN104408707B (en) | Rapid digital imaging fuzzy identification and restored image quality assessment method | |
| Ahn et al. | Deep blind video quality assessment based on temporal human perception | |
| CN111340749B (en) | Image quality detection method, device, equipment and storage medium | |
| US11310475B2 (en) | Video quality determination system and method | |
| CN112950502B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN112906488A (en) | Security protection video quality evaluation system based on artificial intelligence | |
| Da et al. | Perceptual quality assessment of nighttime video | |
| CN113298764B (en) | High-speed camera imaging quality analysis method based on image noise analysis | |
| Van Damme et al. | A full-and no-reference metrics accuracy analysis for volumetric media streaming | |
| CN111160340B (en) | Moving object detection method and device, storage medium and terminal equipment | |
| Sonawane et al. | Image quality assessment techniques: An overview | |
| Oh et al. | Blind sharpness prediction based on image-based motion blur analysis | |
| Regis et al. | Video quality assessment based on the effect of the estimation of the spatial perceptual information | |
| Dakkar et al. | VStab-QuAD: A New Video-Stabilization Quality Assessment Database | |
| CN114640815A (en) | Video processing method and device, electronic equipment and storage medium | |
| Mello et al. | A Comparative Study of Objective Video Quality Assessment Metrics. | |
| Li et al. | A document image dataset for quality assessment | |
| CN111179317A (en) | Interactive teaching system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |