CN118214851A - Video quality evaluation method and system - Google Patents
Video quality evaluation method and system Download PDFInfo
- Publication number
- CN118214851A CN118214851A CN202410432471.3A CN202410432471A CN118214851A CN 118214851 A CN118214851 A CN 118214851A CN 202410432471 A CN202410432471 A CN 202410432471A CN 118214851 A CN118214851 A CN 118214851A
- Authority
- CN
- China
- Prior art keywords
- loss
- video
- quality
- transmission
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 76
- 230000005540 biological transmission Effects 0.000 claims abstract description 135
- 238000004364 calculation method Methods 0.000 claims abstract description 68
- 230000015572 biosynthetic process Effects 0.000 claims description 32
- 238000003786 synthesis reaction Methods 0.000 claims description 32
- 238000001303 quality assessment method Methods 0.000 claims description 23
- 238000012952 Resampling Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 15
- 238000011156 evaluation Methods 0.000 claims description 10
- 230000002194 synthesizing effect Effects 0.000 claims description 10
- 239000000203 mixture Substances 0.000 claims description 9
- 230000004927 fusion Effects 0.000 description 24
- 238000004891 communication Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 239000002131 composite material Substances 0.000 description 3
- 230000000750 progressive effect Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention provides a video quality evaluation method and a video quality evaluation system. Wherein the method comprises: when the image quality loss exists in the video encoding and decoding and transmission processes, carrying out loss calculation to obtain the quality loss; transmitting the quality loss to a receiving side; the receiving side calculates a first transmission loss between a preceding stage and a present stage; the video quality is evaluated based on the quality loss and the first transmission loss as required. The video quality evaluation method and the system thereof creatively integrate the full-reference quality evaluation algorithm into the video quality evaluation method on the basis of the cooperation of the transmitting side and the receiving side, break through the limitation that viewers have no original video data and no reference information, and greatly improve the accuracy of quality evaluation.
Description
Technical Field
The invention relates to the technical field of video communication, in particular to a video quality evaluation method and a video quality evaluation system.
Background
In video communication systems, it is often necessary to collect playing quality data of video for operation and maintenance or subsequent optimization, so a video quality evaluation method capable of providing objective metrics to accurately reflect video viewing experience is a very important ring in video communication systems.
The quality evaluation methods in the prior art generally fall into three categories:
The full-reference quality evaluation method uses an original video before processing as a reference and compares the original video with a processed video. The quality of the video is assessed by comparing the information amounts or feature similarities of the two. This approach is relatively mature and generally provides accurate assessment results. However, this method is only suitable for the test stage where the receiving side can obtain the original video in advance, and cannot be applied to the actual scene. For example, in a practical application scenario, the receiving side generally cannot directly obtain the original video that is not transmitted, because the receiving side can only receive the compressed video stream after being transmitted.
The half-reference quality evaluation method evaluates using partial characteristic information of the distorted image and the original image, instead of complete information of the entire image. This will result in limitations in the evaluation results because some full image level information is missing and the overall quality of the video cannot be fully reflected.
The no-reference quality evaluation method uses only the distorted image itself, without any reference information. The evaluation algorithm directly analyzes the features of the image to calculate the video quality. Due to the lack of reference information, no-reference quality assessment methods often fail to give objective assessment results. In the reference-free network video evaluation method as proposed in patent CN202210428037.9, in order to reflect image quality more accurately, side information such as coding quantization parameter, code rate, etc. is introduced for comprehensive evaluation. However, the accuracy of the method has a large improvement space, and particularly for the situation of video synthesis transcoding, the quality evaluation by using the quantization parameter as a characteristic can only reflect the last synthesis loss and cannot reflect the accumulated coding loss; the accuracy and feasibility of reference-free quality assessment by using image texture features are relatively challenging.
Disclosure of Invention
The invention provides a video quality evaluation method and a system thereof, which creatively fuses a full-reference quality evaluation algorithm when evaluating video quality by realizing end-to-end cooperation at a transmitting side and a receiving side in an actual application scene, so that an evaluation process can simultaneously utilize original video data, thereby breaking through the limitation that viewers have no original video data and no reference information, and remarkably improving the accuracy of video quality evaluation.
In a first aspect, the present invention provides a video quality evaluation method, which is characterized in that the method includes:
When the image quality loss exists in the video encoding and decoding and transmission processes, carrying out loss calculation to obtain the quality loss;
transmitting the quality loss to a receiving side;
the receiving side calculates a first transmission loss between a preceding stage and a present stage;
The video quality is evaluated based on the quality loss and the first transmission loss as required.
In a second aspect, the present invention also provides a video quality evaluation system, which is characterized in that the system includes: a computing device, a transmitting device, a receiving side and an evaluating device; wherein the method comprises the steps of
The computing device is used for carrying out loss computation when the image quality loss exists in the video encoding and decoding process and the transmission process, so as to obtain the quality loss;
the transmitting means is configured to transmit the quality loss to the receiving side;
the receiving side is used for calculating a first transmission loss between a previous stage and a current stage;
The evaluation means is for evaluating the video quality based on the quality loss and the first transmission loss according to demand.
The invention provides a video quality evaluation method and a system thereof: firstly, comprehensively considering the image quality loss in the video coding and decoding and transmission process, carrying out quality evaluation by using a full reference quality algorithm in the video coding and decoding and transmission process, and sending the losses as additional information to a receiving side, so that a more accurate video quality evaluation result can be obtained, namely, carrying out real-time full reference quality evaluation at a loss generation end, and breaking through the restriction that the receiving side cannot acquire original video data or reference information; secondly, the application scene of multi-stage coding can be better processed, in multi-stage coding, video can usually pass through multiple coding and decoding processes, the coding and decoding of each stage can influence the video quality, and the quality change from end to end can be embodied by carrying out quality evaluation on an execution end, so that the video quality problem can be better analyzed and understood, the video quality of the whole coding transmission link can be better monitored and optimized, and the watching experience of an end user can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a video quality evaluation method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a video quality assessment system provided by an embodiment of the present invention;
fig. 3 is a video quality evaluation application scene diagram provided by an embodiment of the present invention;
Fig. 4 is another video quality evaluation application scene diagram provided by an embodiment of the present invention.
Detailed Description
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Summary of The Invention
As described above, the invention provides a video quality evaluation method and a system thereof, which creatively integrate a full-reference quality evaluation algorithm into a transmitting side and a receiving side on the basis of cooperation, thereby breaking through the limitation that viewers have no original video data and no reference information, and greatly improving the accuracy of quality evaluation.
Exemplary method
Fig. 1 is a flowchart of a video quality evaluation method according to an embodiment of the present invention, where the embodiment includes the following steps:
s101: and when the image quality loss exists in the video encoding and decoding and transmission processes, carrying out loss calculation to obtain the quality loss.
Wherein the quality loss includes a service loss and a second transmission loss, which is a transmission loss between stages other than the stage on the receiving side in the transmission process. The service loss includes coding loss and/or synthesis loss.
The invention does not evaluate the quality problem of the original video, does not consider the acquisition distortion condition, considers that the acquired original video has high quality, stable frame rate and resolution, does not have abnormal brightness and contrast, and does not have abnormal working conditions such as blurring, color cast, noise and the like. The coding loss of the transmitting side, the coding loss of the video transcoding server during transcoding, the synthesis loss of the video transcoding server during synthesis resampling and the end-to-end transmission loss are analyzed in an important way.
Irrespective of the partial decoding situation at the receiving side, a network retransmission or error correction mechanism is required to ensure that each decoded frame at the receiving side is a complete image frame. The failure of image decoding requires a subsequent re-encoding of the I-frame for recovery. Each frame of image played is a perfect decoding frame, and no error concealment technology exists, so that in the case of multi-stage transmission, the quality of the decoding image of the present stage is consistent with that of the intermediate decoding image at the encoding position of the previous stage. Therefore, the intermediate decoded image can be obtained when the image of the previous stage is encoded, and the intermediate decoded image and the corresponding image to be encoded are subjected to contrast calculation coding loss, so that the image to be encoded of the previous stage is not required to be transmitted to the current stage to utilize the contrast calculation coding loss of the image to be encoded and the decoded image of the current stage as described in the full reference quality evaluation method in the prior art, and the limitation that the full reference quality evaluation method in the prior art can only be applied in a test stage but can not be applied in an actual scene is solved.
Specifically, step S101 includes, but is not limited to, steps S201 to S203:
S201: and carrying out loss calculation in video coding to obtain the coding loss.
Specifically, the image to be encoded is encoded, and an intermediate decoding image is directly derived, wherein the intermediate decoding image is intermediate data in the encoding process of the encoder, and the intermediate decoding image can be directly derived by adopting a technical means to avoid decoding performance consumption; and calculating the intermediate decoded image and the image to be encoded based on a full-reference quality evaluation algorithm to obtain the encoding loss.
Specifically, the coding loss QAenc (n) is calculated according to the following formula:
QAenc(n)＝Menc(F(n),Fenc(n))
Where F is the image to be encoded, i.e. the encoder input image, fenc is the intermediate decoded image derived from the encoder, n is the number of the video stream, and function Menc represents the full reference quality assessment algorithm. The full reference quality assessment algorithm includes, but is not limited to, SSIM (an index for assessing the degree of similarity between two images), VIF (an index for assessing the difference in visual information between two images), and FSIM (an assessment index based on the local features of the images). For example, by SSIM calculation, the coding loss result interval is [0,1], and a larger coding loss value indicates a smaller distortion.
As an alternative embodiment, as shown in fig. 3, the whole process includes two ends of a transmitting side and a receiving side, the transmitting side has a coding loss, the coding loss of the transmitting side is calculated by applying the calculation formula QAenc (n) = Menc (F (n), fenc (n)), where F is an acquired original image, and Fenc is a corresponding intermediate decoded image derived from an encoder.
As another alternative embodiment, as shown in fig. 4, the whole process includes three ends of a transmitting side, a video transcoding server and a receiving side, in contrast to the previous embodiment, in addition to the encoding loss of the transmitting side as in the previous embodiment, the encoding loss of the video transcoding server also exists in this embodiment, the encoding loss of the video transcoding server is calculated by applying the calculation formula, where F is a downsampled image of each sub-picture, fenc is an intermediate decoded image derived from the encoder for the corresponding sub-picture, and then the encoding loss of each sub-picture is weighted to obtain the encoding loss of the whole synthesized picture. Although the composite picture is encoded during encoding, the encoding loss can be calculated for each sub-picture.
S202: and carrying out loss calculation in video synthesis to obtain the synthesis loss.
When the transcoding server needs to perform multi-picture composition, there may be resampling of the image and thus a composition loss.
Specifically, the transcoding server (n levels) decodes images in the multi-path video from n-1 levels respectively to obtain decoded images of the sub-pictures; resampling the decoded image of the sub-picture to obtain a resampled image of the sub-picture; comparing and calculating the resampled image of the sub-picture with the decoded image of the corresponding sub-picture to obtain the resampling loss of each sub-picture; synthesizing the resampled images of all the sub-pictures to obtain a synthesized picture; and weighting and calculating the resampling loss of each sub-picture to obtain the synthesis loss.
More specifically, the resampling loss QAsyn (n) for each sub-picture is calculated according to the following formula:
QAsyn(n)＝QA(Fdec(n),Fs(n))
where n is the number of the sub-picture, fs (n) is the resampled image of the sub-picture, fdec (n) is the decoded image of the corresponding sub-picture, and QA represents the resampled loss calculation function.
As shown in fig. 4, in the case of image composition, the video transcoding server performs downsampling, assuming that the composite picture Fmix is made up of N pictures, each of the sub-pictures received from the transmitting side is denoted by F (N), where n=1, 2,.
S203: and carrying out loss calculation in video transmission to obtain the second transmission loss.
The sending side may need to perform network transmission to the video transcoding server, which may be affected by network jitter. Considering that the first transmission loss between the receiving side and its preceding stage and the second transmission loss between the other stages in the transmission process except for the receiving side stage differ only in whether or not the receiving side is involved in the calculation at the receiving side, the generation principle of both are identical, and the calculation method is the same, so for convenience of the subsequent description of the calculation method thereof, the first transmission loss and the second transmission loss are collectively referred to as transmission loss. The transmission loss calculation step specifically comprises the following steps:
Specifically, n-1 level calculates the time sequence of the video specific frame to obtain n-1 level time sequence; the n-1 level sends the coded image and the n-1 level time sequence to n levels; the n-level decodes the n-1-level coded image, and calculates the time sequence of the specific frame to obtain an n-level time sequence; and calculating the n-1 level time sequence and the n level time sequence to obtain transmission loss.
Wherein the time series is a series of data points arranged in time order. Typically, these data points are collected or recorded over successive time intervals. The time sequence in the invention is the time sequence when each video frame is coded and the time sequence when each video frame is decoded.
By "n-1 level computing the time series of a particular frame of video" is meant that in n-1 level (a hierarchy or processing stage) a time series of data is generated based on the characteristics or properties of the particular frame in the video. This includes the encoded time sequence of a particular frame.
This time series data (and corresponding encoded images) is then sent to the next level, i.e., n-level. And in n stages, decoding the coded image sent from the n-1 stages, decoding the specific frame, and calculating the decoded time sequence again to obtain an n-stage time sequence. By comparing the time series of n-level and n-1 level, the transmission loss between these two processing phases can be calculated.
In this process, "transmission loss" refers to loss of information during image or data transmission and processing, which is due to degradation of data quality or loss of information caused by network jitter. The degree of this loss can be quantified by calculating the difference between the n-1 and n-level time series, which is important for optimizing video transmission, storage and processing flows.
The transmission loss QAfluent, which is a correlation coefficient of two time series, is calculated according to the following formula:
Qs, qr are the n-1 level time series and the n level time series, respectively, cov (Qs, qr) represents the covariance of the two time series, and σ Qs and σ Qr represent the standard deviation of the two sequences, respectively.
The correlation coefficient of two time series is a statistic that measures the strength and direction of the linear relationship between two variables, ranging in value from-1 to 1.
When the correlation coefficient is 1, the existence of a complete positive linear relation between the two variables is indicated;
When the correlation coefficient is-1, the existence of a complete negative linear relationship between the two variables is indicated;
When the correlation coefficient is 0, it means that there is no linear relationship between the two variables.
The closer the correlation coefficient is to 1, the smaller the difference between the two time series is, the smaller the transmission loss is, and the higher the video smoothness is.
For example, as shown in fig. 3, the transmitting side calculates a time series of certain frames for a period of time, and the receiving side collects the time series of corresponding frames with the transmitting side encoding output frame rate as a standard. And then a second transmission loss from the transmitting side to the receiving side is obtained by calculation of the transmission loss formula described above.
As another example, as shown in fig. 4, the video transcoding server collects the time series of the corresponding frames with the transmission side encoding output frame rate as a standard, for the time series of the specific frames for a period of time calculated by the transmission side. And then calculating and obtaining the first transmission loss from the transmitting side to the video transcoding server through the transmission loss formula. Then the video transcoding server calculates the time sequence of the specific frames of the period of time, and the receiving side collects the time sequence of the corresponding frames by taking the encoding output frame rate of the video transcoding server as a standard. And then calculating and obtaining the second transmission loss from the video transcoding server to the receiving side through the transmission loss formula. The specific frames used for transmission loss in the two processes may be the same or different.
S102: the quality loss is transmitted to the receiving side.
The quality loss is transmitted to the receiving side as additional information, so that the limit that the receiving side does not have original video data and reference information is broken through, and the accuracy of quality evaluation is greatly improved.
S103: the receiving side calculates a first transmission loss between the preceding stage and the present stage.
S104: the video quality is evaluated based on the quality loss and the first transmission loss as required.
In addition, the invention can optimize the corresponding operation process of each end based on the coding loss and/or the synthesis loss of each end; and optimizing the corresponding transmission process based on the end-to-end transmission loss. And each item of loss data is respectively presented, so that more comprehensive data support can be provided for operation and follow-up optimization of the video communication system, a system administrator is helped to quickly locate and solve the quality problem, and the reliability and stability of the whole video service are improved.
The coding loss, the synthesis loss and the transmission loss can also be subjected to fusion calculation to obtain an overall loss, and the overall loss is presented to a receiving side user to obtain an integrated video quality evaluation. Such comprehensive assessment may help the user to better understand the quality of video service and thus make more accurate decisions. For example, in real-time video communication or online video streaming services, a user may adjust his viewing behavior based on the magnitude of the overall loss, such as selecting a more appropriate video resolution or adjusting network bandwidth settings to optimize the viewing experience.
Specifically, the step of calculating the overall loss may be: and respectively calculating the receiving loss of each level and the output loss of each level step by step according to the video stream transmission sequence so as to obtain the final total loss.
As an alternative embodiment, the reception losses of each stage are calculated according to the following formula:
QArn＝αQAsn-1+(1-α)QAfluentn-1～n
wherein QAs n-1 is n-1 level output loss, QAfluent n-1～n is transmission loss between n-1 level and n level, alpha is an adjustment coefficient between 0 and 1, the size of the adjustment coefficient alpha depends on the application scene, and the higher the frame rate is, the larger alpha is. When the frame rate is high, the transmission loss QAfluent n-1～n caused by small fluctuation has little influence on the experience of a viewer, and alpha takes a larger value; when the frame rate is relatively low, the viewer is relatively sensitive to the frame rate, and α takes a small value.
Calculating the output loss of each stage according to the following formula:
QAsn＝Fu(QArn,QAn)
where QA n generates losses for n stages themselves and Fu is a peer loss fusion function.
For example, as shown in fig. 3, the transmission side has a reception loss QAr 1 of 0 and an output loss QAs 1 of QAenc 1, which is a coding loss generated by itself, as the first stage; the receiving side is the last stage, and has a receiving loss of QAr 2＝αQAs1+(1-α)QAfluent1～2, which does not generate loss, so the output loss QAs 2 of the receiving side is the receiving loss QAr 2, i.e. the final total loss is QAr 2.
For example, as shown in fig. 4, the transmission side has a reception loss QAr 1 of 0 and an output loss QAs 1 of QAenc 1, which is a coding loss generated by itself, as the first stage; the video transcoding server is used as a second stage, the receiving loss is QAr 2＝αQAs1+(1-α)QAfluent1～2, and the output loss QAs 2 is the fusion value QAs 2＝Fu(QAr2,QAenc2,QAsyn2 of the receiving loss QAr 2 of the video transcoding server and the encoding loss QAenc 2 and the synthesizing loss QAsyn 2 generated by the video transcoding server; the receiving side is the last stage, and has a receiving loss of QAr 3＝αQAs2+(1-α)QAfluent2～3, which does not generate loss, so the output loss QAs 3 of the receiving side is the receiving loss QAr 3, i.e. the final total loss is QAr 3. The fusion function Fu in the video transcoding server output loss QAs 2 calculation formula can be adjusted through experiments, and the following formula is referred to:
Wherein Wmix is the pixel area of the synthesized picture, ws (n) is the pixel area of the nth sub-picture in the synthesized picture, QAr 2 (n) is the video transcoding server receiving loss corresponding to the nth sub-picture in the synthesized picture, QAenc 2 (n) is the coding loss of the nth sub-picture in the video transcoding server, and QAsyn 2 (n) is the synthesis loss of the nth sub-picture in the video transcoding server.
As another alternative embodiment, when the n-1 level self-generated loss and the n level self-generated loss have the same kind of loss, the same kind of loss is fused in a same kind, and then the same kind of fusion result is fused with other kinds of loss. Compared with the method that all types of losses are fused together, the method has the advantages that the same type of losses are fused first, so that the calculation complexity can be reduced, and the calculation efficiency is improved.
For example, as shown in fig. 4, both the transmitting side and the video transcoding server have coding loss, that is, similar loss, the transmitting side coding loss and the video transcoding server coding loss are fused, and then the result QAenc is fused with the second transmission loss QAfluent 1～2 between the transmitting side and the video transcoding server and the composite loss QAsyn 2 of the video transcoding server, so as to obtain the output loss QAs 2 of the video transcoding server.
Exemplary System
Correspondingly, the embodiment of the invention also provides a video quality evaluation system. Fig. 2 is a block diagram of a video quality evaluation system according to an embodiment of the present invention, and as shown in fig. 2, a system 100 according to the present embodiment includes: a computing device 101, a transmitting device 102, a receiving side 103, and an evaluating device 104; wherein the method comprises the steps of
The computing device 101 is configured to perform loss computation when there is an image quality loss in the video encoding and decoding process and obtain a quality loss;
The transmitting means 102 is configured to transmit the quality loss to the receiving side 103;
the receiving side 103 is configured to calculate a first transmission loss between a preceding stage and a present stage;
The evaluation means 104 is for evaluating the video quality based on the quality loss and the first transmission loss according to requirements.
The quality loss includes a service loss and a second transmission loss, which is a transmission loss between stages other than the stage of the receiving side 103 in the transmission process.
The service loss includes coding loss and/or synthesis loss.
The computing device 101 includes:
An encoding unit 105 for performing loss calculation at the time of video encoding, obtaining the encoding loss;
A synthesizing unit 106 for performing loss calculation at the time of video synthesis, obtaining the synthesis loss; and/or
A transmission unit 107 for performing loss calculation at the time of video transmission, to obtain the second transmission loss.
The encoding unit 105 includes:
A module for encoding the image to be encoded and directly deriving an intermediate decoded image;
And the encoding calculation module 108 is used for calculating the intermediate decoded image and the image to be encoded to obtain encoding loss.
The transmission unit 107 includes:
a module for calculating a time sequence of a video specific frame through n-1 levels to obtain an n-1 level time sequence;
For transmitting its encoded image through the n-1 stage and the n-1 stage time series to n stages;
The n-level decoding unit is used for decoding the n-1 level coded image through the n levels and calculating the time sequence of the specific frame to obtain an n-level time sequence;
And a transmission calculation module 109, configured to calculate the n-1 level time series and the n level time series, so as to obtain a transmission loss.
The synthesizing unit 106 includes:
a module for decoding images in the multi-path video from n-1 level respectively to obtain decoded images of the sub-picture;
A module for resampling the decoded image of the sub-picture to obtain a resampled image of the sub-picture;
The sampling calculation module 110 is configured to perform a contrast calculation on the resampled image of the sub-picture and the decoded image of the corresponding sub-picture, so as to obtain a resampling loss of each sub-picture;
a module for synthesizing the resampled images of each sub-picture to obtain a synthesized picture;
and a module for weighting and calculating the resampling loss of each sub-picture to obtain the synthesis loss.
The intermediate decoded image is intermediate data of the encoding process.
The encoding calculation module 108 is further configured to calculate the intermediate decoded image and the image to be encoded based on a full reference quality evaluation algorithm, so as to obtain an encoding loss.
The encoding calculation module 108 is further configured to calculate the encoding loss QAenc (n) according to the following formula:
QAenc(n)＝Menc(F(n),Fenc(n))
Wherein F is the image to be encoded, fenc is the intermediate decoded image, n is the number of the video stream, and function Menc represents the full reference quality assessment algorithm.
The full reference quality assessment algorithm includes SSIM, VIF, and FSIM.
The transmission calculation module 109 is further configured to calculate the transmission loss QAfluent according to the following formula:
wherein Qs, qr are the n-1 level time series and the n level time series, respectively, cov (Qs, qr) represents the covariance of the two time series, and σ Qs and σ Qr represent the standard deviation of the two sequences, respectively.
The sampling calculation module 110 is further configured to calculate a resampling loss QAsyn of each sub-picture according to the following formula:
QAsyn(n)＝QA(Fdec(n),Fs(n))
where n is the number of the sub-picture, fs (n) is the resampled image of the sub-picture, fdec (n) is the decoded image of the corresponding sub-picture, and QA represents the resampled loss calculation function.
The system 100 further comprises an optimizing means 111, the optimizing means 111 comprising:
Means for optimizing a corresponding operational procedure for each end based on the coding loss and/or the synthesis loss for each end;
and means for optimizing a corresponding transmission procedure based on the end-to-end transmission loss.
The evaluation device 104 comprises a fusion unit 112 for fusion calculating the coding loss, the synthesis loss and the transmission loss, obtaining an overall loss.
The fusion unit 112 includes a step-by-step calculation module 113 for calculating the receiving loss of each level and the output loss of each level step by step according to the video stream transmission sequence.
The step-by-step calculation module 113 is configured to calculate the reception loss QAr n at each stage according to the following formula:
QArn＝αQAsn-1+(1-α)QAfluentn-1～n
Wherein QAs n-1 is n-1 stage output loss, QAfluent n-1～n is transmission loss between n-1 and n stages, and α is an adjustment coefficient between 0 and 1;
The step-by-step calculation module 113 is further configured to calculate the output loss QAs n of each stage according to the following formula:
QAsn＝Fu(QArn,QAn)
where QA n generates losses for n stages themselves and Fu is a peer loss fusion function.
The magnitude of the adjustment coefficient alpha depends on the application scene, and the higher the frame rate is, the larger the alpha is.
The fusion unit 112 further includes a module for performing similar fusion on the similar losses when the n-1 level self-generated losses and the n level self-generated losses exist, and then fusing the similar fusion result with other similar losses.
It should be noted that although the operations of the video quality assessment method of the present invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in that particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
Furthermore, although several devices, units, or modules of a video quality assessment system are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more modules described above may be embodied in one module in accordance with embodiments of the present invention. Conversely, the features and functions of one module described above may be further divided into a plurality of modules to be embodied.
While the spirit and principles of the present invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments nor does it imply that features of the various aspects are not useful in combination, nor are they useful in any combination, such as for convenience of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
The invention provides:
1. A method for evaluating video quality, the method comprising:
When the image quality loss exists in the video encoding and decoding and transmission processes, carrying out loss calculation to obtain the quality loss;
transmitting the quality loss to a receiving side;
the receiving side calculates a first transmission loss between a preceding stage and a present stage;
The video quality is evaluated based on the quality loss and the first transmission loss as required.
2. The video quality evaluation method according to claim 1, wherein the quality loss includes a service loss and a second transmission loss, the second transmission loss being a transmission loss between stages other than the stage on the receiving side in the transmission process.
3. The video quality evaluation method according to claim 2, wherein the service loss includes coding loss and/or composition loss.
4. The video quality evaluation method according to claim 3, wherein the step of calculating loss when there is a loss of image quality in the video codec and transmission process, the step of obtaining the loss of quality is specifically: performing loss calculation in video coding to obtain coding loss;
performing loss calculation during video synthesis to obtain the synthesis loss; and/or
And carrying out loss calculation in video transmission to obtain the second transmission loss.
5. The video quality evaluation method according to claim 4, wherein the step of performing loss calculation at the time of video encoding to obtain the encoding loss is specifically:
coding the image to be coded and directly deriving an intermediate decoded image;
and calculating the intermediate decoded image and the image to be encoded to obtain the encoding loss.
6. The video quality evaluation method according to claim 4, wherein the transmission loss includes the first transmission loss and the second transmission loss, and the calculation step of the transmission loss specifically includes:
calculating the time sequence of a specific frame of the video by n-1 level to obtain n-1 level time sequence;
The n-1 level sends the coded image and the n-1 level time sequence to n levels;
The n-level decodes the n-1-level coded image, and calculates the time sequence of the specific frame to obtain an n-level time sequence;
and calculating the n-1 level time sequence and the n level time sequence to obtain transmission loss.
7. The video quality evaluation method according to claim 4, wherein the step of performing loss calculation at the time of video composition to obtain the composition loss is specifically:
respectively decoding images in the multi-path video from n-1 level to obtain decoded images of the sub-picture;
resampling the decoded image of the sub-picture to obtain a resampled image of the sub-picture;
Comparing and calculating the resampled image of the sub-picture with the decoded image of the corresponding sub-picture to obtain the resampling loss of each sub-picture;
synthesizing the resampled images of all the sub-pictures to obtain a synthesized picture;
And weighting and calculating the resampling loss of each sub-picture to obtain the synthesis loss.
8. The video quality evaluation method according to any one of claims 5 to 7, wherein the intermediate decoded image is intermediate data of an encoding process.
9. The video quality evaluation method according to any one of claims 5 to 7, wherein the step of calculating the intermediate decoded image and the image to be encoded to obtain the coding loss specifically comprises: and calculating the intermediate decoded image and the image to be encoded based on a full-reference quality evaluation algorithm to obtain the encoding loss.
10. The video quality evaluation method according to claim 9, characterized in that the coding loss QAenc (n) is calculated according to the following formula:
QAenc(n)＝Menc(F(n),Fenc(n))
Wherein F is the image to be encoded, fenc is the intermediate decoded image, n is the number of the video stream, and function Menc represents the full reference quality assessment algorithm.
11. The video quality assessment method according to claim 9, wherein the full reference quality assessment algorithm comprises SSIM, VIF and FSIM.
12. The video quality evaluation method according to item 6, characterized in that the transmission loss QAfluent is calculated according to the following formula:
wherein Qs, qr are the n-1 level time series and the n level time series, respectively, cov (Qs, qr) represents the covariance of the two time series, and σ Qs and σ Qr represent the standard deviation of the two sequences, respectively.
13. The video quality evaluation method according to claim 7, wherein the resampling loss QAsyn of each sub-picture is calculated according to the following formula:
QAsyn(n)＝QA(Fdec(n),Fs(n))
where n is the number of the sub-picture, fs (n) is the resampled image of the sub-picture, fdec (n) is the decoded image of the corresponding sub-picture, and QA represents the resampled loss calculation function.
14. The video quality evaluation method according to any one of claims 1 to 7, characterized in that the method further comprises, after the step of evaluating the video quality based on the quality loss and the first transmission loss according to demand: optimizing a corresponding operational procedure for each end based on the coding loss and/or the synthesis loss for each end;
And optimizing the corresponding transmission process based on the end-to-end transmission loss.
15. The video quality evaluation method according to any one of claims 1 to 7, characterized in that the step of evaluating the video quality based on the quality loss and the first transmission loss on demand is specifically: and carrying out fusion calculation on the coding loss, the synthesis loss and the transmission loss to obtain the total loss.
16. The video quality evaluation method according to claim 15, wherein the step of performing fusion calculation of the coding loss, the synthesis loss, and the transmission loss to obtain an overall loss is specifically: and respectively calculating the receiving loss of each level and the output loss of each level step by step according to the video stream transmission sequence.
17. The video quality evaluation method according to claim 16, wherein the reception loss QAr n at each stage is calculated according to the following formula:
QArn＝αQAsn-1+(1-α)QAfluentn-1～n
Wherein QAs n-1 is n-1 stage output loss, QAfluent n-1～n is transmission loss between n-1 and n stages, and α is an adjustment coefficient between 0 and 1;
the output loss QAs n for each stage is calculated according to the following formula:
QAsn＝Fu(QArn,QAn)
where QA n generates losses for n stages themselves and Fu is a peer loss fusion function.
18. The video quality evaluation method according to claim 17, wherein the magnitude of the adjustment coefficient α depends on an application scene, and the higher the frame rate, the larger the α.
19. The method according to any one of claim 15, wherein when there is a similar loss between the n-1 level self-generated loss and the n level self-generated loss, the similar loss is first subjected to similar fusion, and then the similar fusion result is fused with other similar losses.
20. A video quality assessment system, the system comprising: a computing device, a transmitting device, a receiving side and an evaluating device; wherein the method comprises the steps of
The computing device is used for carrying out loss computation when the image quality loss exists in the video encoding and decoding process and the transmission process, so as to obtain the quality loss;
the transmitting means is configured to transmit the quality loss to the receiving side;
the receiving side is used for calculating a first transmission loss between a previous stage and a current stage;
The evaluation means is for evaluating the video quality based on the quality loss and the first transmission loss according to demand.
21. The video quality evaluation system according to claim 20, wherein the quality loss includes a service loss and a second transmission loss, the second transmission loss being a transmission loss between stages other than the stage on the receiving side in the transmission process.
22. The video quality assessment system according to claim 21, wherein the service loss comprises coding loss and/or composition loss.
23. The video quality assessment system according to claim 22, wherein the computing device comprises:
an encoding unit for performing loss calculation at the time of video encoding to obtain the encoding loss;
A synthesizing unit for performing loss calculation at the time of video synthesis to obtain the synthesis loss; and/or
And the transmission unit is used for carrying out loss calculation in video transmission to obtain the second transmission loss.
24. The video quality evaluation system according to claim 23, wherein the encoding unit includes:
A module for encoding the image to be encoded and directly deriving an intermediate decoded image;
and the coding calculation module is used for calculating the intermediate decoded image and the image to be coded to obtain coding loss.
25. The video quality evaluation system according to claim 23, wherein the transmission unit includes:
a module for calculating a time sequence of a video specific frame through n-1 levels to obtain an n-1 level time sequence;
For transmitting its encoded image through the n-1 stage and the n-1 stage time series to n stages;
The n-level decoding unit is used for decoding the n-1 level coded image through the n levels and calculating the time sequence of the specific frame to obtain an n-level time sequence;
and the transmission calculation module is used for calculating the n-1 level time sequence and the n level time sequence to obtain transmission loss.
26. The video quality evaluation system according to claim 23, wherein the synthesizing unit includes:
a module for decoding images in the multi-path video from n-1 level respectively to obtain decoded images of the sub-picture;
A module for resampling the decoded image of the sub-picture to obtain a resampled image of the sub-picture;
the sampling calculation module is used for comparing and calculating the resampled image of the sub-picture with the decoded image of the corresponding sub-picture to obtain the resampling loss of each sub-picture;
a module for synthesizing the resampled images of each sub-picture to obtain a synthesized picture;
and a module for weighting and calculating the resampling loss of each sub-picture to obtain the synthesis loss.
27. The video quality evaluation system according to any one of claims 24-26, wherein the intermediate decoded image is intermediate data of an encoding process.
28. The video quality evaluation system according to any one of claims 24-26, wherein the encoding calculation module is further configured to calculate the intermediate decoded image and the image to be encoded based on a full reference quality evaluation algorithm, to obtain an encoding loss.
29. The video quality evaluation system of claim 28, wherein the encoding calculation module is further configured to calculate the encoding loss QAenc (n) according to the formula:
QAenc(n)＝Menc(F(n),Fenc(n))
Wherein F is the image to be encoded, fenc is the intermediate decoded image, n is the number of the video stream, and function Menc represents the full reference quality assessment algorithm.
30. The video quality assessment system according to claim 28, wherein the full reference quality assessment algorithm comprises SSIM, VIF and FSIM.
31. The video quality assessment system according to claim 25, wherein the transmission calculation module is further configured to calculate the transmission loss QAfluent according to the following formula:
wherein Qs, qr are the n-1 level time series and the n level time series, respectively, cov (Qs, qr) represents the covariance of the two time series, and σ Qs and σ Qr represent the standard deviation of the two sequences, respectively.
32. The video quality assessment system according to claim 26, wherein said sample calculation module is further configured to calculate the resampling loss QAsyn for each sub-picture according to the following formula:
QAsyn(n)＝QA(Fdec(n),Fs(n))
where n is the number of the sub-picture, fs (n) is the resampled image of the sub-picture, fdec (n) is the decoded image of the corresponding sub-picture, and QA represents the resampled loss calculation function.
33. The video quality assessment system according to any one of claims 20-26, wherein the system further comprises an optimizing means comprising:
Means for optimizing a corresponding operational procedure for each end based on the coding loss and/or the synthesis loss for each end;
and means for optimizing a corresponding transmission procedure based on the end-to-end transmission loss.
34. The video quality evaluation system according to any one of claims 20 to 26, wherein the evaluation means includes a fusion unit for performing fusion calculation of the coding loss, the synthesis loss, and the transmission loss to obtain an overall loss.
35. The video quality evaluation system according to claim 34, wherein the fusion unit includes a progressive calculation module for respectively performing progressive calculation on each level of reception loss and each level of output loss according to a video stream transmission order.
36. The video quality evaluation system according to claim 35, wherein the progressive calculation module is configured to calculate the respective stage reception losses QAr n according to the following formula:
QArn＝αQAsn-1+(1-α)QAfluentn-1～n
Wherein QAs n-1 is n-1 stage output loss, QAfluent n-1～n is transmission loss between n-1 and n stages, and α is an adjustment coefficient between 0 and 1;
the stage-by-stage calculation module is further configured to calculate the output loss QAs n of each stage according to the following formula:
QAsn＝Fu(QArn,QAn)
where QA n generates losses for n stages themselves and Fu is a peer loss fusion function.
37. The video quality evaluation system according to claim 36, wherein the magnitude of the adjustment coefficient α depends on an application scene, and the higher the frame rate, the larger the α.
38. The video quality evaluation system according to claim 34, wherein the fusion unit further comprises a module for, when there is a similar loss between the n-1 level self-generated loss and the n level self-generated loss, firstly performing similar fusion on the similar loss, and then fusing the similar fusion result with other similar losses.
Claims (10)
1. A method for evaluating video quality, the method comprising:
When the image quality loss exists in the video encoding and decoding and transmission processes, carrying out loss calculation to obtain the quality loss;
transmitting the quality loss to a receiving side;
the receiving side calculates a first transmission loss between a preceding stage and a present stage;
The video quality is evaluated based on the quality loss and the first transmission loss as required.
2. The video quality evaluation method according to claim 1, wherein the quality loss includes a service loss and a second transmission loss, the second transmission loss being a transmission loss between stages other than the stage on the receiving side in the transmission process.
3. The video quality assessment method according to claim 2, wherein the service loss comprises coding loss and/or synthesis loss.
4. The method for evaluating video quality according to claim 3, wherein the step of calculating loss when there is a loss of image quality in the video codec and transmission process, the step of obtaining the loss of quality is specifically: performing loss calculation in video coding to obtain coding loss;
performing loss calculation during video synthesis to obtain the synthesis loss; and/or
And carrying out loss calculation in video transmission to obtain the second transmission loss.
5. The method for evaluating video quality according to claim 4, wherein the step of performing loss calculation at the time of video encoding to obtain the encoding loss is specifically:
coding the image to be coded and directly deriving an intermediate decoded image;
and calculating the intermediate decoded image and the image to be encoded to obtain the encoding loss.
6. The video quality evaluation method according to claim 4, wherein the transmission loss includes the first transmission loss and the second transmission loss, and the calculation step of the transmission loss is specifically:
calculating the time sequence of a specific frame of the video by n-1 level to obtain n-1 level time sequence;
The n-1 level sends the coded image and the n-1 level time sequence to n levels;
The n-level decodes the n-1-level coded image, and calculates the time sequence of the specific frame to obtain an n-level time sequence;
and calculating the n-1 level time sequence and the n level time sequence to obtain transmission loss.
7. The method for evaluating video quality according to claim 4, wherein the step of calculating a loss at the time of video composition, the step of obtaining the composition loss is specifically:
respectively decoding images in the multi-path video from n-1 level to obtain decoded images of the sub-picture;
resampling the decoded image of the sub-picture to obtain a resampled image of the sub-picture;
Comparing and calculating the resampled image of the sub-picture with the decoded image of the corresponding sub-picture to obtain the resampling loss of each sub-picture;
synthesizing the resampled images of all the sub-pictures to obtain a synthesized picture;
And weighting and calculating the resampling loss of each sub-picture to obtain the synthesis loss.
8. The video quality evaluation method according to any one of claims 5 to 7, wherein the intermediate decoded image is intermediate data of an encoding process.
9. The video quality evaluation method according to any one of claims 5 to 7, wherein the step of calculating the intermediate decoded image and the image to be encoded to obtain a coding loss is specifically: and calculating the intermediate decoded image and the image to be encoded based on a full-reference quality evaluation algorithm to obtain the encoding loss.
10. A video quality assessment system, the system comprising: a computing device, a transmitting device, a receiving side and an evaluating device; wherein the method comprises the steps of
The computing device is used for carrying out loss computation when the image quality loss exists in the video encoding and decoding process and the transmission process, so as to obtain the quality loss;
the transmitting means is configured to transmit the quality loss to the receiving side;
the receiving side is used for calculating a first transmission loss between a previous stage and a current stage;
The evaluation means is for evaluating the video quality based on the quality loss and the first transmission loss according to demand.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410432471.3A CN118214851A (en) | 2024-04-10 | 2024-04-10 | Video quality evaluation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410432471.3A CN118214851A (en) | 2024-04-10 | 2024-04-10 | Video quality evaluation method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118214851A true CN118214851A (en) | 2024-06-18 |
Family
ID=91450356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410432471.3A Pending CN118214851A (en) | 2024-04-10 | 2024-04-10 | Video quality evaluation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118214851A (en) |
-
2024
- 2024-04-10 CN CN202410432471.3A patent/CN118214851A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9014279B2 (en) | Method, system and apparatus for enhanced video transcoding | |
| Maia et al. | A concise review of the quality of experience assessment for video streaming | |
| US8031770B2 (en) | Systems and methods for objective video quality measurements | |
| JP5436458B2 (en) | Multi-view image encoding method, multi-view image decoding method, multi-view image encoding device, multi-view image decoding device, multi-view image encoding program, and multi-view image decoding program | |
| EP2649801B1 (en) | Method and apparatus for objective video quality assessment based on continuous estimates of packet loss visibility | |
| KR101783071B1 (en) | Method and apparatus for assessing the quality of a video signal during encoding or compressing of the video signal | |
| JP2015501568A (en) | Scene change detection for perceptual quality assessment in video sequences | |
| Joskowicz et al. | A parametric model for perceptual video quality estimation | |
| Duanmu et al. | Assessing the quality-of-experience of adaptive bitrate video streaming | |
| Takeuchi et al. | Perceptual quality driven adaptive video coding using JND estimation | |
| Leszczuk et al. | Key indicators for monitoring of audiovisual quality | |
| US6755531B2 (en) | Motion picture code evaluator and billing system | |
| KR20100071803A (en) | Method for restoring transport error included in image and apparatus thereof | |
| CN118214851A (en) | Video quality evaluation method and system | |
| EP2736261A1 (en) | Method For Assessing The Quality Of A Video Stream | |
| JP4573301B2 (en) | Video signal frame synchronization method | |
| US20060067410A1 (en) | Method for encoding and decoding video signals | |
| Issa et al. | Estimation of time varying QoE for high definition IPTV distribution | |
| US8493449B2 (en) | Method of estimating video quality at any resolution | |
| Shi et al. | A user-perceived video quality assessment metric using inter-frame redundancy | |
| Riker et al. | A hybrid prediction and assessment quality of experience approach for videostreaming applications over wireless mesh networks | |
| Chin et al. | Bitstream-based quality metric for packetized transmission of H. 264 encoded video | |
| Kwon et al. | A novel video quality impairment monitoring scheme over an ipty service with packet loss | |
| Zhang et al. | QoE Models for Online Video Streaming | |
| Lee et al. | Hybrid NR Video Quality Metric with Decodable Payload |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |