CN100559881C - A kind of method for evaluating video quality based on artificial neural net - Google Patents

Abstract

A kind of method for evaluating video quality based on artificial neural net belongs to the computer digit field of video processing.The spatial character of this evaluation algorithms by analyzing video image (fuzzy, entropy, blocking effect, frequency domain energy spectrometer, saturation) and time response (frame-to-frame differences) are come the extent of damage of computed image.The chrominance space saturation can effectively not improved the evaluation effect of algorithm as there being one of parameter with reference to evaluation algorithms.Native system is based on artificial neural net design, so the realization of algorithm comprises the training process and the test process of network.For selected training sample (sequence of video images), at first extract six above-mentioned parameters, and obtain the desired output (subjective assessment result) of training sample in training by subjective assessment.The characteristic parameter of training sample is imported artificial neural net with corresponding subjective assessment result as training parameter.Show that according to experiment the evaluation result that this objective evaluation system is obtained is consistent with the visual experience height of human eye.

Description

A kind of method for evaluating video quality based on artificial neural net

Technical field

The present invention relates to the computer digit field of video processing, designed a kind of no reference number method for evaluating video quality based on artificial neural net.

Background technology

Video image is the main forms of visual visual information, and it is one of present important field of research that video image is handled.Each link in the video image application as collection, compression, transmission, processing, storage etc., all can have influence on picture quality inevitably.Because image is finally watched for the user, therefore the correct evaluation for picture quality is one of key technology in the image information engineering.

The evaluation method of video image quality can be divided into two big classes: subjective assessment and objective evaluation.Subjective assessment is directly to utilize the observer to determined a kind of test of picture quality by the visually-perceptible of altimetric image, its measurement result is reliable, and the actual observation quality that meets image, therefore, occupy critical role in the image quality measurement field, at present existing corresponding international standard (as ITU-R BT.500, ITU-R BT.710).But the test environment of subjective assessment requires very strict, the implementation procedure complexity, and do not have repeatability.Along with the continuous expansion of video image application, under some applied environment, can't carry out effective subjective assessment.And the objective evaluation algorithm is set up Mathematical Modeling by analyzing video processnig algorithms and human visual system, to realize the automatic measurement to picture quality.This type of algorithm can be embedded into each link of Video processing, to video flow quality carry out quantitative analysis and can further analyze with the Adjustment System parameter to improve final image display effect.Therefore, picture quality objective evaluation algorithm is a present research focus.

Traditional evaluating objective quality algorithm is representative with MSE (mean square error) and PSNR (Y-PSNR).This type of evaluation algorithms is theoretical intuitively, calculating is simple, can obtain measurement result preferably in the ordinary course of things.But this objective evaluation algorithm based on signal is not considered the apperceive characteristic of human eye when watching image, and therefore in some cases, measurement result that is obtained and subjective sensation are also inconsistent.In recent years, fully analyzing on video processnig algorithms and the vision perception characteristic basis, each research institution has proposed a lot of improvement algorithms, and to the degree of dependence of distorted image not, we can be divided into algorithm three kinds according to algorithm:

1. based on the video quality evaluation of full reference frame (FR-Full Reference)

Undistorted raw video image can be used for the objective evaluation algorithm of picture quality fully, and therefore, measurement result is relatively accurate.But have following problem: (1) has considered that the evaluation algorithms data volume of original video is huge, the computational complexity height; (2) in a lot of application scenarios, undistorted image can not obtain at receiving terminal;

2. based on the video quality evaluation that reduces reference frame (RR-Reduced Reference)

The characteristic parameter of undistorted raw video image can be used in the objective evaluation algorithm of picture quality.This algorithm at first defines and extracts the various features parameter of raw video image, by a narrower auxiliary channel of bandwidth these parameters is transferred to receiving terminal then.Same characteristic parameter at receiving terminal extraction damaged image to be measured by the statistical analysis to this two stack features parameter, obtains the objective evaluation result of damaged image quality.The FR algorithm of comparing, the required original image information of this algorithm reduces in a large number, and implementation complexity reduces; Simultaneously, owing to considered original image, this algorithm still has higher confidence level.But have following problem: (1) is analyzed theoretically, and the statistics of some impaired video images and corresponding undistorted image are identical probably, but differ widely on subjective vision; (2) statistics owing to image is easy to change with the change of factors such as viewing distance, brightness range, even obtains diverse statistics, and this type of algorithm can't be considered this situation fully; (3) in some applied environment, can't increase the characteristic parameter that auxiliary channel transmits undistorted image.

3. based on the video quality evaluation of no reference frame (NR-No Reference)

Need not the information of any original image, by extracting the various features parameter of damaged image, seek such as the mpeg block edge, point-like noise or image deflects such as image blurring are obtained the evaluation result of picture quality.These class methods are without any need for the data from raw video image, only can realize mass measurement to impaired video image at receiving terminal, and requirement and subjective assessment result have consistency preferably, and therefore, the research difficulty of this type of algorithm is bigger.Owing to do not need original image information, do not have to put on a plurality of links of Video Applications system and carry out on-line testing with reference to the objective evaluation algorithm.Native system has also proposed the digital video image Objective Quality Assessment system of the no reference of a cover just on this basis.

Nothing is with reference to the domestic and international present Research of algorithm

1. domestic present Research

People such as Wang Xindai have proposed a kind of no-reference video quality evaluation algorithms in 2004, can better be applied to the quality evaluation of video traffics such as wireless and IP.This method is by extracting the watermark that is embedded in advance in the compressed video, and compares with the original watermark copy of receiving terminal storage, thereby realizes the evaluation to video quality.Because this system need add watermark information in primary signal, can destroy the quality of image to a certain extent, in addition, also needs original watermark information as a comparison in receiving end, therefore has certain application limitations.

People such as Yin Xiaoli have proposed a kind of non-reference picture quality appraisement method based on half fragility digital watermarking algorithm in 2006.The result is better in this system evaluation, but have as above-mentioned same problem, and this algorithm is based on rest image in addition, can't handle video.

People such as Wang Zhengyou have proposed a kind of non-reference picture quality appraisement method based on shielding effect in the angle from noise measuring in 2006.At first the Hosaka piecemeal is improved, cancelled of the restriction of this method picture size.By piecemeal, image is made a distinction with different frequency contents, calculate the noise of each height piece then.According to the pollution level of image, the non-reference picture Y-PSNR NPSNR based on shielding effect has been proposed.Experimental result shows that this method has and do not have reference, computational complexity is low, subjective and objective than characteristics such as unanimities.This method is just come the analysis image quality based on noise, but noise mainly is the main damage in the simulated television system, so it is not suitable for digital video.

People such as poplar Fu Zheng have proposed a kind of reference-free quality evaluation method that is applicable to based on the block encoding video in 2006.At first cover and cover characteristic with contrast and proposed a blocking artifact that meets the subjective vision perception and estimate in conjunction with the brightness of human vision, according to the influence of filtering, provided a kind of quality evaluating method that is suitable for using different compressions and Processing Algorithm then based on the block encoding reconstructing video to blocking artifact.Experiment shows that this quality evaluation estimates with the subjective quality evaluation consistency is preferably arranged.But this algorithm is only analyzed the blocking effect in the image, and parameter obviously is the total quality that is not enough to reflect image.And this algorithm is based on human eye and covers characteristic, and will be correctly also be unusual difficulty with covering characteristic formulism.

2. foreign study present situation

People such as Pina Marziliano have proposed a kind of nothing of the video image of analyzing based on fog-level with reference to evaluation method in 2002, this method is that the extended area of image border is analyzed, and sets threshold of perception current according to subjective feeling.This method computation complexity is low, and speed is near real-time broadcasting speed.Because the assess performance of this method will depend on the effect of rim detection, so the application of algorithm will be subjected to certain limitation.

The non-reference picture quality appraisement method that people such as Hanghang Tong proposed a kind of JPEG2000 of being applied in 2004, by marginal points all in the image being classified as impaired or not having impaired, use PCA that given edge pixel point is extracted local feature then, to judge that it is whether fuzzy or ringing effect is arranged.Also use the whether impaired extent of damage of judging topography of edge pixel point simultaneously, can be applied to all kinds of local features.The effect of this method also depends on the effect of rim detection to a great extent, but does not also have very perfect edge detection algorithm at present, so this algorithm effects also will be restricted.And this algorithm also is merely able to analyze at rest image, and can not handle video.

People such as Remco Muijs have proposed a kind of non-reference picture quality appraisement method based on signature analysis in 2005.Mainly the blocking effect in the image is analyzed, because it is the key factor of influence based on piecemeal compressed image quality.By the position of detecting piece and the degree that observability is come the decision block effect.This algorithm is to carry out breakdown diagnosis according to the position of block edge, but the spatial deviation of image will cause the skew of block edge, and makes result of calculation inaccurate.And this method only analyzed a kind of damage in the image, i.e. blocking effect, and obviously it is the oeverall quality that is not enough to reflect image.

Other external correlative studys also comprise by regression algorithm calculates subjective and objective fitting parameter and realizes that by the training of human artificial neural networks nothing is with reference to the objective evaluation algorithm.But all lack multianalysis to the video image characteristic parameter.Algorithm exists certain limitation more at present, can't provide the total quality of image or video, and also can be subjected to various restrictions in application.

Summary of the invention

In order to overcome the deficiency of present evaluation method, the present invention has designed a kind of no-reference video quality evaluating method based on artificial neural net.This evaluation method is taken all factors into consideration the various features parameter of video image, and gathers as the input of objective evaluation system in conjunction with the visual characteristic definition and the extraction designated parameter of human eye, and the subjective assessment result of definition correspondence is as the output set of evaluation system.By selecting test sample book, and the subjective assessment result who obtains test sample book realizes the training to neural net.Experiment shows, the artificial neural net after the training can obtain and the subjective assessment consistent objective evaluation result of height as a result.

Technical thought of the present invention is characterised in that:

1, a kind of no-reference video quality evaluating method has been proposed.This method can be come the computed image extent of damage by the room and time statistical property of analyzing video image.These characteristics comprise: space characteristics (fuzzy.Entropy, blocking effect, frequency domain energy spectrometer, saturation) and time response (frame-to-frame differences).

2, considered the damage of chrominance space.Monochrome information is not occupied an leading position in having with reference to the evaluation algorithms design at present, and chrominance space almost is left in the basket.The colourity damage that native system is analyzed can effectively improve evaluation result.

3, analyze every characteristic of video image and making up, as input parameter based on the evaluation system of artificial neural net.

System framework of the present invention is referring to Fig. 1.Because this system is based on the artificial neural net design, therefore comprise two major parts of training and testing.We are referred to as " training sample " video image of participation training, are used for the video image called after " test sample book " of detection system performance.This method comprises the steps: successively

1), for training sample, we at first extract the characteristic parameter (leaching process is as shown in Figure 2) of video image, as the input parameter of artificial neural net; Simultaneously, obtain the subjective assessment result of training sample by subjective assessment, as the desired output result of artificial neural net;

2), deposit above-mentioned getting parms in parameter database, be used for the training of human artificial neural networks;

3), the training process of executor's artificial neural networks (training process of neural net as shown in Figure 3): for each input sample, we are with the input as system of six characteristic parameters of its correspondence, correspond respectively to six input nodes of artificial neural net, and should import the desired output of the desired output (subjective testing result) of sample correspondence as system.According to the difference value of desired output with actual output, promptly error is adjusted each internodal connection weights in the artificial neural net.

4), when satisfying termination condition (native system with the accumulated error of each all samples of training as controlled condition, accumulated error in certain training is less than specified threshold value, then training process finishes), promptly set up with reference to evaluation model based on the nothing of artificial neural net and to finish.Afterwards, the sample that can pass a test detects the systematic function of this model; The appointment of threshold value is generally 10 ^-4, but concrete numerical value depends on the number of samples and the characteristic of participating in training, in different applications, the training stop condition need be according to the actual conditions adjustment.

5), for the input test sample book, extract same characteristic parameter (as shown in Figure 2), be input to then in the model that has trained, can obtain quality evaluation result (test process is as shown in Figure 5) to test sample book.

Explanation for each module in Fig. 1 and the above-mentioned steps is then carried out according to the order of figure numbering.

Characteristic extracting module: as shown in Figure 2, video image for input, at first extract nine characteristic parameters, be respectively image activity, average gradient, edge energy statistics, zero-crossing rate statistics, entropy, blocking effect, frequency domain Energy distribution characteristic, eight spatial character parameters of saturation and time response parameter of frame-to-frame differences.Wherein, except saturation was the characteristic parameter of statistics chrominance space, other parameters were all from the brightness space of video image.In addition, we average weighting with preceding four spatial characters, are defined as the fuzzy parameter of image.

Image active A ctivity _Image: be the characteristic quantity that mainly reflects the image detail texture, be the image brightness first-order difference statistics of image in level and vertical direction.Earlier each row of image is made forward difference, and calculates its quadratic sum,, and then statisticses of all row are added up, obtain this horizontal direction image activity, be shown below as the statistics of this row:

$D\left(i\right)=\underset{j=1}{\overset{M-1}{\mathrm{\Σ}}}{(Y(i,j)-Y(i,j-1))}^2,(0\≤i\≤N-1)---\left(1\right)$

$H_1=\underset{i=0}{\overset{N-1}{\mathrm{\Σ}}}D\left(i\right)---\left(2\right)$

Wherein, Y is the brightness space of image, and (i is that image i is capable, the pairing brightness value of j row pixel j) to Y.M, N be respectively level and vertical direction number of pixels (in the following formula, variable Y (i, j), the definition of M, N is all identical therewith).D (i) is the capable statistics of i, H ₁Be the image activity of horizontal direction, the active V of image that uses the same method and to obtain vertical direction ₁, both additions can be obtained the image activity (Activity of entire image _Image=H ₁+ V ₁).

Average gradient Ave_Gradient: be the characteristic quantity of reflection image detail texture, obtain by finding the solution the pixel second differnce.Specifically be calculated as follows, ask each pixel second differnce in the horizontal and vertical directions earlier, ask quadratic sum then, divided by 2, calculate square root again, the Grad to all pixels adds up at last, and, obtain the average gradient value of entire image divided by total pixel count.Formula is as follows:

$G(i,j)=\sqrt{\frac{{\▿}_j^{2}Y(i,j)+{\▿}_i^{2}Y(i,j)}{2}}---\left(3\right)$

$\mathrm{Ave}\_\mathrm{Gradient}=\frac{\underset{i=1}{\overset{N-2}{\mathrm{\Σ}}}\underset{j=1}{\overset{M-2}{\mathrm{\Σ}}}G(i,j)}{(M-2)\×(N-2)}---\left(4\right)$

Wherein, With Be respectively image pixel (i, the j) gradient of the level of position and vertical direction, following calculating:

${\▿}_{j}Y(i,j)=(Y(i,j-1)-Y(i,j))-(Y(i,j)-Y(i,j+1))---\left(5\right)$

${\▿}_{i}Y(i,j)=(Y(i-1,j)-Y(i,j))-(Y(i,j)-Y(i+1,j))---\left(6\right)$

Edge energy statistics Edge_Energy: a very important feature is exactly the edge of image characteristic in the image, its feature is mild along the pixel variation of edge trend, and change perpendicular to the pixel of edge trend violent, according to this characteristic, edge that can statistical picture.The defined formula of edge energy statistics is as follows,

e(i，j)＝E ₁(Y(i，j))+E ₂(Y(i，j)) (7)

$\mathrm{Edge}\_\mathrm{Energy}=\frac{1}{(M-2)\×(N-2)}\underset{i=1}{\overset{N-2}{\mathrm{\Σ}}}\underset{j=1}{\overset{M-2}{\mathrm{\Σ}}}{e}^2(i,j)---\left(8\right)$

Wherein, E ₁And E ₂Be two 3*3 templates, concrete value: ${\mathrm{<figure-callout\; id="1"\; label="E"\; filenames="C20081010613200202.png"\; state="\{\{state\}\}">E</figure-callout>}}_1=\left|\begin{array}{ccc}-1& -1& 1\\ -1& 4& -1\\ 1& -1& -1\end{array}\right|,$ ${\mathrm{<figure-callout\; id="2"\; label="E"\; filenames="C20081010613200202.png"\; state="\{\{state\}\}">E</figure-callout>}}_2=\left|\begin{array}{ccc}1& -1& -1\\ -1& 4& -1\\ -1& -1& 1\end{array}\right|.$

Zero-crossing rate statistics ZC: be the characteristic quantity of reflection image border details, the symbol by more adjacent two first-order differences obtains, and adds up respectively for the zero-crossing rate of level and vertical direction.Zero-crossing rate with horizontal direction is calculated as example, specifically be calculated as follows: the brightness value to each row pixel carries out first-order difference earlier, and if the symbol of more adjacent then two first-order differences is two adjacent first-order difference contrary signs, then the statistical value of zero-crossing rate just is 1, otherwise is 0.Following formula,

$Z_h=\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{M-2}{\mathrm{\&Sigma;}}}{z}_h(i,j)---\left(10\right)$

Wherein, Z _hBe the zero-crossing rate statistic of horizontal direction, the zero-crossing rate statistic Z that uses the same method and to obtain vertical direction _vBoth additions can be obtained the zero-crossing rate statistic (ZC=Z of image _h+ Z _v).

These four parameters average weighting, can obtain first spatial parameter of image: fuzzy parameter Blur _Image(see figure 2).Computing formula is as follows:

${\mathrm{Blur}}_{\mathrm{image}}=\frac{{\mathrm{Activity}}_{\mathrm{image}}+\mathrm{Ave}\_\mathrm{Gradient}+\mathrm{Edge}\_\mathrm{Energy}+\mathrm{ZC}}{4}---\left(11\right)$

Entropy Entropy _Image: an index of the contained amount of information size of reflection image, the entropy solution formula in computational methods such as the information theory.Monochrome information is found the solution the following formula of entropy,

${\mathrm{Entropy}}_{\mathrm{image}}=\underset{i=1}{\overset{L}{\mathrm{\&Sigma;}}}p\left(x_i\right){\log }_2\frac{1}{p\left(x_i\right)}---\left(12\right)$

Wherein, L represents the number of the gray scale that occurs, p (x _i) represent the probability (number of times that definition of probability occurs for this gray scale is divided by the total pixel number of image) of this gray scale, the Entropy that obtains at last _ImageIt is exactly the entropy of entire image.

Blocking effect Block _Image: in the coding and decoding video algorithm, the pseudo-edge that blocking effect caused shows as 8 to be signal pseudoperiod in cycle more, therefore can come blocking effect is analyzed from frequency spectrum.With the horizontal direction is example, at first calculate the first-order difference of every row and ask absolute value, and as follows to the capable solution formula of i,

f _i(j)＝|Y(i，j+1)-Y(i，j)| 0≤j≤M-2 (13)

The if block effect is apparent in view, then the capable signal f of i _i(j) will have in one pseudoperiod signal, the cycle is 8.To f _i(j) carry out zero padding, making its length is 2 integral number power, obtains f ' _i(j), then it is carried out fast fourier transform, and calculate the amplitude spectrum of fourier coefficient, more same operation is carried out in each provisional capital, at last the amplitude spectrum of the Fourier transform of all row is added up.So just obtained the array of the frequency domain statistics of a horizontal direction, as shown in the formula

$F\left(n\right)=\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\mathrm{FFT}\left(f_i^{\&prime;}\left(j\right)\right),0\&le;n\&le;L-1---\left(14\right)$

If L is the length after the zero padding, F (n) to be analyzed and can find, the if block effect is apparent in view, and then (position L/2) is peak value will occur on the characteristic frequency position for L/8, L/4 at 1/8 multiple of F (n) length.Blocking effect is obvious more, and the peak value of characteristic frequency point is big more, and promptly the cycle is that the intensity of 8 signal is strong more.Because the appearance of these peak values mainly is because blocking effect causes, so that reflection spatially is the gap at edge between the adjacent block is big more.When computing block effect degree, can on the position that these peak values may occur, analyze, to determine that at first it is a peak value, promptly this value is greater than the value of the right and left; If peak value then carries out medium filtering at this point, the window of filtering can be 3 or 5, peak value is deducted the value that obtains behind the medium filtering DC coefficient (F (0)) divided by amplitude spectrum, promptly obtains reflecting the blocking effect characteristic value B in the horizontal direction of image _hMasking characteristics if decide like this in the consideration human-eye visual characteristic.Because after first-order difference, take absolute value,, show that the correlation between this delegation's pixel is more little so DC coefficient is big more, the variation that is image is big more, according to the visual characteristic of human eye, in the many places of marginal information, be the violent place of image change, damage just is not easy to be discovered.Though,, and just can better react later the visual characteristic of human eye divided by DC coefficient owing to relatively more violent the becoming of the interior pixel variation of its piece is not easy to be discovered so the difference of adjacent tile edges is the same.

Same method is used for vertical direction can obtains the characteristic value B of blocking effect in vertical direction _v, the characteristic value that on average is total blocking effect is asked in both additions:

${\mathrm{Block}}_{\mathrm{image}}=\frac{B_h+B_v}{2}---\left(15\right)$

The Fourier transformation computation amount is bigger owing to each provisional capital is carried out, and compares time-consuming, so done further improvement on this basis.Still be example with the horizontal direction, in improving algorithm, not simply the first-order difference of every row to be carried out fast fourier transform, but after the first-order difference of several rows added up, again the array that adds up is carried out fast fourier transform, and calculate the amplitude spectrum of fourier coefficient.Position at the L/8 of amplitude spectrum is that the characteristic frequency position is analyzed by the method for being introduced before, obtains the horizontal direction blocking effect characteristic value of these several row.Ensuing several row are done same operation, and the characteristic value addition of all horizontal directions is asked on average, be blocking effect characteristic value in the horizontal direction.Several row are added up handle, it is capable to be generally 8-16, can reflect the blocking effect degree in a zone, and delegation just not only, this is more reasonable for the masking characteristics in the human-eye visual characteristic.Can obtain the characteristic value of blocking effect vertical direction after the same method.Method after the improvement improves a lot on speed, and better effects if.In experiment, we carry out fast fourier transform after 16 first-order differences of going are added up again.

Frequency domain energy spectrometer Fre_Energy: in conjunction with human visual system (HSV) characteristic, consider that human eye has different sensitivitys for frequency components different in the image, extract the frequency domain Energy distribution feature of present image based on the contrast sensitivity function (CSF) of people such as Mannos and Sakrison foundation.Can be by analyzing the CSF indicatrix with the multi-stage bandpass filter, therefore, in the process of extracting the image frequency domain Energy distribution, can be earlier one group of directive band pass filter with picture breakdown, each filter be only made response to spatial frequency and direction near certain zone its centre frequency.In computed image frequency domain energy, generally obtain the DCT of frequency domain parameter, conversion such as FFT all can be used for the frequency domain character of analysis image, but because the data structure of wavelet decomposition and the multichannel characteristic of visually-perceptible have great similitude, thereby adopt wavelet transformation as analysis tool here.Concrete steps are as follows:

Luminance picture is carried out the level Four wavelet transformation, and shown in Fig. 4 (a), in this article, employed small echo is the W53 small echo, its high-pass coefficient and low-pass coefficients be respectively 0.25 ,-0.5,0.25}, 0.125,0.25,0.75,0.25 ,-0.125}.Amplitude to the wavelet coefficient after changing is added up respectively, promptly tries to achieve the quadratic sum with the wavelet amplitude of one-level, and divided by the total sample of this one-level, and the value that obtains is the ENERGY E (Lx) on the frequency band of this one-level correspondence, and wherein the x value is 0,1,2,3,4.Utilize the non-linear bandpass characteristics of CSF then, the wavelet coefficient of different spaces frequency band after the wavelet decomposition is weighted, weighted value is the mean value of CSF curve in frequency band, sees Fig. 4 (b).The frequency domain energy calculation as shown in the formula, Fre_Energy is the frequency domain energy distributions feature that obtains image.

Fre_Energy＝2.25×E(L0)+2.87×E(L1)+3.16×E(L2)+2.56×E(L3)+1.00×E(L4) (16)

In this process, what resulting value reflected is the shared ratio of composition of human eye sensitivity in the image, as image (component that has only lowest frequency) for pure color, perhaps be the image (the frequency domain energy almost concentrates on the part of high frequency entirely) of noise entirely, human eye is to see what interested content, does not also have sensitive composition certainly.

Saturation Chrome _Image: the damage of picture quality on color shows as the decline of colourity saturation more.Specifically be calculated as follows, utilize the mould value of two chromatic components of image pixel, promptly calculate the quadratic sum of U and V component and ask its square root, the statistics of all mould values that add up again, and, promptly obtain the saturation of colourity divided by the number of chromatic component.Be shown below,

${\mathrm{Chrome}}_{\mathrm{image}}=\frac{1}{M_{\mathrm{UV}}\&times;N_{\mathrm{UV}}}\underset{i=0}{\overset{N_{\mathrm{UV}}-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{M_{\mathrm{UV}}-1}{\mathrm{\&Sigma;}}}\sqrt{U^2(i,j)+V^2(i,j)}---\left(17\right)$

Wherein, U (i, j) and V (i is respectively that image i is capable, the pairing colourity value of j row carrier chrominance signal j), M _UV, N _UVBe that carrier chrominance signal is at level and vertical direction pixel count order.For four kinds of common chroma, the relation of the value of these two variablees and M, N is as follows:

When sample format is 4:4:4, M _UV=M, N _UV=N;

When sample format is 4:2:2, $M_{\mathrm{UV}}=\frac{\mathrm{<figure-callout\; id="2"\; label="M"\; filenames="C20081010613200202.png"\; state="\{\{state\}\}">M</figure-callout>}}{2},$ N _UV＝N；

When sample format is 4:1:1, $M_{\mathrm{UV}}=\frac{M}{4},$ N _UV＝N；

When sample format is 4:2:0, $M_{\mathrm{UV}}=\frac{\mathrm{<figure-callout\; id="2"\; label="M"\; filenames="C20081010613200202.png"\; state="\{\{state\}\}">M</figure-callout>}}{2},$ $N_{\mathrm{UV}}=\frac{\mathrm{<figure-callout\; id="2"\; label="N"\; filenames="C20081010613200202.png"\; state="\{\{state\}\}">N</figure-callout>}}{2};$

Frame-to-frame differences Diff_Frame: the extraction of temporal characteristics, this is important parameters very in the video image quality evaluation.This feature is found the solution poor based on the monochrome information of front and back two width of cloth images, as shown in the formula,

$\mathrm{Diff}\_\mathrm{Frame}=\frac{\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{M-1}{\mathrm{\&Sigma;}}}|Y(i,j,t+1)-Y(i,j,t)|}{M\&times;N}---\left(18\right)$

Wherein, variable t is the time shaft parameter of video sequence.This parametric solution of frame-to-frame differences be the average information of the luminance difference absolute value of back one frame and former frame respective pixel.

Six characteristic parameters of the training sample of characteristic extracting module output simultaneously, in training process, also need to provide the subjective assessment result of these training samples, as the desired output data of system in the training stage as the input parameter of artificial neural net.

Description of drawings

Fig. 1 is the system block diagram of video quality evaluation system.

Fig. 2 is the schematic diagram of characteristic parameter extraction module.

Fig. 3 is based on the artificial neural net block diagram of BP algorithm.

Fig. 4 is a schematic diagram of determining the corresponding weight coefficient of wavelet decomposition gained different frequency bands according to the CSF curve.Wherein, Fig. 4 (a) is the different spaces frequency band of level Four small echo after changing and the schematic diagram of corresponding weight coefficient, and Fig. 4 (b) be the schematic diagram that the CSF curve defines in corresponding space frequency band weighted value.

Fig. 5 is the flow chart that cycle tests carries out objective evaluation.

Fig. 6 is based on the main interface of the video evaluation algorithm of artificial neural net.

Fig. 7 is the training sample schematic diagram.

Fig. 8 is the subjective and objective fitness analysis of training sample in test process, and wherein abscissa is the objective evaluation result of system's output, and ordinate is the subjective assessment result.

Fig. 9 is the test sample book schematic diagram.

Figure 10 is the subjective and objective fitness analysis of test sample book in test process, and wherein abscissa is the objective evaluation result of system's output, and ordinate is the subjective assessment result.

Embodiment

In the system block diagram of Fig. 1, all from the video test sequence of standard, choosing of this type of sequence should be in strict accordance with the standard of ITU-R BT.1210, particularly training sample for training sample and test sample book.The sequence storage can be adopted YUV file and other canonical forms.The subjective assessment algorithm adopts is that two in the international standard stimulate continuous mass scale method, with DMOS (Difference Mean Opinion Score) as the required subjective assessment result of sample.In addition, need explanation a bit, during system framework in front and each module are introduced, our two field picture is as a basic processing unit, but for interleaved video display format, the base unit that calculates is one, only above-mentioned processing is carried out in the field, top of a frame picture and get final product in processing procedure.

Video quality evaluation is mainly realized by software, comprises the quality evaluation of characteristic parameter extraction, network training, list entries etc.Below by the implementation procedure of the objective evaluation of high definition television video (hereinafter to be referred as " HD video ") being described in detail system.

The multilayer neural network that is based on the BP learning algorithm that native system adopts is called for short the BP network here.

The artificial neural net training module: as shown in Figure 3, the BP network of native system design only contains a hidden layer; Six inputs of input layer definition node corresponds respectively to six characteristic parameters; Output layer has only a node, the evaluation result of corresponding video quality, and in the training stage, this node is a desired output, i.e. the subjective assessment result of corresponding training sample.Here the conventional training algorithm (steepest gradient descent method) of the BP network using of definition is introduced here no longer in detail.The artificial neural net that trains can be realized the objective evaluation to video image quality, and for the cycle tests of input, the input test module is to detect the performance of this evaluation system.

Test module: testing process as shown in Figure 5.For the cycle tests of input, concrete performing step is as follows:

501: open test sequence file, prepare reads image data;

502: read present frame F _CWith next frame F _NView data;

503: extract previously defined six characteristic parameters;

504: each characteristic parameter is sued for peace respectively, for the character pair parameter of calculating whole sequence is prepared;

505: judge F _NWhether frame is the last frame of this sequence, if change 506 over to; If not, change 502 over to;

506: find the solution the average of each characteristic parameter according to the totalframes of sequence, promptly obtain the final characteristic parameter of this sequence correspondence;

507: with the input value of above-mentioned six sequential parameter values finding the solution, by the objective evaluation result of the network calculations list entries that trained as artificial neural net;

508: the objective evaluation result who exports this cycle tests video quality.

Fig. 6 is system master interface, can realize the browsing of preview, video various features CALCULATION OF PARAMETERS, parameter database of video to be measured, and realizes the operations such as objective evaluation to cycle tests.

At first we select to satisfy eight sections original HD videos (yuv format, 4:2:0,10～14 seconds) of test request, and obtain 8M, 10M, 12M, 14M, 16M, 18M, 20M and 25M totally eight compressed bit streams by MPEG2 hardware compression device.Add undistorted original series, we have foundation and test that 72 HD videos are used for system.

Secondly, we obtain the subjective assessment result of these high definition sequences by the continuous mass scale method of two stimulations.For training sample, the subjective assessment result is used for the adjustment model parameter as the desired output in the model training process; For test sample book, the subjective assessment result is used for the objective evaluation result's of analytical system output accuracy, promptly by calculating subjective and objective result's coefficient correlation, judges the assess performance of this system.

Training process: we select all code checks of six sequences as training sample set (this set is totally 54 training samples), and the content of six training sequences as shown in Figure 7.Extract each sample six parameters, deposit parameter database in together with the subjective assessment result of correspondence, be used for the training of neural net.In the artificial neural net design, the node number of hidden layer is defined as 16, factor of momentum 0.95, and the initial value 0.0001 of learning rate (in training, can adjust), the initial value that connects weights between node is a random number.When satisfying the training end condition (threshold value of definition cumulative errors is 0.0004 in realizing at present), model training is finished.For the system that trains, we import training sample once more, and the measurement result of analytical system output and subjective assessment result's fitting degree, and as shown in Figure 8, the coefficient correlation of two groups of data is 0.958.

Test process: our definition has neither part nor lot in two other video of training and all compression bit rates thereof as test sample book collection (totally 18 test sample books), and the content of two cycle testss as shown in Figure 9.Similarly, six parameters of first abstraction sequence, the output evaluation result of computing system, i.e. objective evaluation result then.Because these sequences never appear in the training process, therefore, we can come evaluation system reliability in actual applications by this test process.Figure 10 has provided the picture quality for test sample book, the objective evaluation result of system evaluation gained and subjective assessment result's correlation, and promptly coefficient correlation is 0.930.

System advantage is analyzed: the no-reference video quality objective evaluation system based on artificial neural net can be effectively in conjunction with the subjective assessment result with high reliability who generally acknowledges, therefore match human visual system farthest makes objective evaluation result and subjective assessment result have higher correlation.Thereby make this system can replace the subjective assessment of time-consuming, effort, cost costliness to a certain extent, this also is the meaning of objective evaluation.This system is after finishing training, in practical video quality evaluation process without any need for data from raw video image, and only can realize measurement quick and precisely to impaired video quality at receiving terminal, further, this system also can be used as measurement module and is placed on a plurality of intermediate links of Video Applications system and carries out on-line testing, to realize application system planning and the performance evaluation based on video quality.

Claims (3)

1. method for evaluating video quality based on artificial neural net, this method comprises the steps: successively

1), for training sample, at first extract the characteristic parameter of this sample, as the input parameter of artificial neural net; Simultaneously, obtain the subjective assessment result of training sample by subjective assessment, as the desired output result of artificial neural net;

2), get parms, comprise the characteristic parameter and the corresponding subjective assessment result of sample, deposit parameter database in, be used for the training of human artificial neural networks above-mentioned;

3), the training process of executor's artificial neural networks: for each input sample, we are with the input of its characteristic of correspondence parameter as system, correspond respectively to the input node of artificial neural net, and should import the desired output of the subjective assessment result of sample correspondence as system; According to the difference value of desired output, adjust each internodal connection weights in the artificial neural net with actual output;

4), when satisfying termination condition, promptly set up with reference to evaluation model based on the nothing of artificial neural net and to finish; Afterwards, the sample that passes a test detects the systematic function of this model;

5), for the input test sample book, extract same characteristic parameter, be input to then in the model that has trained, promptly obtain quality evaluation result to test sample book;

It is characterized in that the characteristic parameter that extracts video image in the step 1) is specially:

Extract nine characteristic parameters, be respectively image activity, average gradient, edge energy statistics, zero-crossing rate statistics, entropy, blocking effect, frequency domain Energy distribution characteristic, eight spatial character parameters of saturation and time response parameter of frame-to-frame differences; And preceding four spatial character parameters are averaged weighting, be defined as the fuzzy parameter of image;

Image active A ctivity _Image:

Ask the active H of horizontal direction image ₁, be shown below:

$D\left(i\right)=\underset{j=1}{\overset{M-1}{\mathrm{\&Sigma;}}}{(Y(i,j)-Y(i,j-1))}^2,(0\&le;i\&le;N-1)---\left(1\right)$

$H_1=\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}D\left(i\right)---\left(2\right)$

Wherein, Y is the brightness space of image, and (i is that image i is capable, the pairing brightness value of j row pixel j) to Y; M, N are respectively the number of pixels of level and vertical direction; D (i) is the capable statistics of i, H ₁Image activity for horizontal direction; The active V of image that uses the same method and obtain vertical direction ₁, both additions are promptly obtained the image active A ctivity of entire image _Image=H ₁+ V ₁

In the following formula, and variable Y (i, j), the definition of M, N is all same as described above;

Average gradient Ave_Gradient:

Ask earlier each pixel average gradient value G (i, j), the Grad to all pixels adds up then, and divided by total pixel count, obtains the average gradient value Ave_Gradient of entire image, formula is as follows:

$G(i,j)=\sqrt{\frac{{\&dtri;}_j^{2}Y(i,j)+{\&dtri;}_i^{2}Y(i,j)}{2}}---\left(3\right)$

$\mathrm{Ave}\_\mathrm{Gradient}=\frac{\underset{i=1}{\overset{N-2}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{M-2}{\mathrm{\&Sigma;}}}G(i,j)}{(M-2)\&times;(N-2)}---\left(4\right)$

Wherein, With

Be respectively image pixel (i, the j) gradient of the level of position and vertical direction are calculated according to following formula:

${\&dtri;}_{j}Y(i,j)=(Y(i,j-1)-Y(i,j))-(Y(i,j)-Y(i,j+1))---\left(5\right)$

${\&dtri;}_{i}Y(i,j)=(Y(i-1,j)-Y(i,j))-(Y(i,j)-Y(i+1,j))---\left(6\right)$

Edge energy statistics Edge_Energy:

Find the solution the margin signal energy of each pixel earlier, the edge energy of finding the solution entire image again, as shown in the formula,

e(i，j)＝E ₁(Y(i，j))+E ₂(Y(i，j)) (7)

$\mathrm{Edge}\_\mathrm{Energy}=\frac{1}{(M-2)\&times;(N-2)}\underset{i=1}{\overset{N-2}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{M-2}{\mathrm{\&Sigma;}}}{e}^2(i,j)---\left(8\right)$

Wherein, E ₁And E ₂Be two 3*3 templates, concrete value: $E_1=\left|\begin{array}{ccc}-1& -1& 1\\ -1& 4& -1\\ 1& -1& -1\end{array}\right|,$ $E_2=\left|\begin{array}{ccc}1& -1& -1\\ -1& 4& -1\\ -1& -1& 1\end{array}\right|;$

Zero-crossing rate statistics ZC:

The zero-crossing rate of difference calculated level and vertical direction, with the horizontal direction zero-crossing rate is example, be calculated as follows: the brightness value to each row pixel carries out first-order difference earlier, the symbol of more adjacent then two first-order differences, if two adjacent first-order difference contrary signs, then the statistical value of zero-crossing rate just is 1, otherwise is 0;

Following formula,

$Z_h=\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{M-2}{\mathrm{\&Sigma;}}}{z}_h(i,j)---\left(10\right)$

Wherein, Z _hBe the zero-crossing rate statistic of horizontal direction, the zero-crossing rate statistic Z that uses the same method and obtain vertical direction _vBoth additions are promptly obtained the zero-crossing rate statistic ZC=Z of image _h+ Z _v

Above four parameters are averaged weighting, promptly obtain first spatial parameter of image: fuzzy parameter Blur _Image, computing formula is as follows:

${\mathrm{Blur}}_{\mathrm{image}}=\frac{{\mathrm{Activity}}_{\mathrm{image}}+\mathrm{Ave}\_\mathrm{Gradient}+\mathrm{Edge}\_\mathrm{Energy}+\mathrm{ZC}}{4}---\left(11\right)$

Entropy Entropy _Image:

Monochrome information is found the solution entropy, following formula,

${\mathrm{Entropy}}_{\mathrm{image}}=\underset{i=1}{\overset{L}{\mathrm{\&Sigma;}}}p\left(x_i\right){\log }_2\frac{1}{p\left(x_i\right)}---\left(12\right)$

Wherein, L represents the number of the gray scale that occurs, p (x _i) represent the probability of this gray scale, the number of times that the calculating of this probability occurs by gray scale obtains the Entropy that above-mentioned formula obtains divided by the total pixel number of image _ImageIt is exactly the entropy of entire image;

The characteristic value Block of total blocking effect _Image:

Horizontal direction: at first calculate the first-order difference of every row and ask absolute value, as follows to the capable solution formula of i,

f _i(j)＝|Y(i，j+1)-Y(i，j)| 0≤j≤M-2 (13)

The f that following formula obtains _i(j) be that length is the array of M-1, to f _i(j) carry out zero padding, make its length become 2 integral number power, obtain f ' _i(j), then it is carried out fast fourier transform, and calculate the amplitude spectrum of fourier coefficient, more same operation is carried out in each provisional capital; The amplitude spectrum of the Fourier transform of all row is added up, obtained the array of the frequency domain statistics of a horizontal direction, as shown in the formula

$F\left(n\right)=\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\mathrm{FFT}\left(f_i^{\&prime;}\left(j\right)\right),0\&le;n\&le;L-1---\left(14\right)$

Wherein, L is the length after the zero padding;

When computing block effect degree, peak point is carried out medium filtering, the window of filtering is 3 or 5, peak value is deducted the value that obtains behind the medium filtering DC coefficient F (0) divided by amplitude spectrum, promptly obtains reflecting the blocking effect characteristic value B in the horizontal direction of image _h

Same method is used for vertical direction obtains the characteristic value B of blocking effect in vertical direction _v, the characteristic value that on average is total blocking effect is asked in both additions:

${\mathrm{Block}}_{\mathrm{image}}=\frac{B_h+B_v}{2}---\left(15\right)$

Frequency domain energy spectrometer Fre_Energy: luminance picture is carried out the frequency domain energy spectrometer obtain image frequency domain energy distributions feature;

Saturation Chrome _Image: two chrominance spaces by image calculate this feature of saturation, be calculated as follows, the quadratic sum of two chromatic component U of computed image pixel and V and ask its square root at first, colourity mould value as pixel, the statistics of all mould values that add up again, and, promptly obtain the saturation parameters of image divided by the number of chromatic component; Be shown below,

${\mathrm{Chrome}}_{\mathrm{image}}=\frac{1}{M_{\mathrm{UV}}\&times;N_{\mathrm{UV}}}\underset{i=0}{\overset{N_{\mathrm{UV}}-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{M_{\mathrm{UV}}-1}{\mathrm{\&Sigma;}}}\sqrt{U^2(i,j)+V^2(i,j)}---\left(17\right)$

Wherein, U (i, j) and V (i is respectively that image i is capable, the pairing colourity value of j row carrier chrominance signal j), M _UV, N _UVBe carrier chrominance signal at level and vertical direction pixel count order, for four kinds of common chroma, the relation of the value of these two variablees and M, N is as follows:

When sample format is 4:4:4, M _UV=M, N _UV=N;

When sample format is 4:2:2, $M_{\mathrm{UV}}=\frac{M}{2},$ N _UV＝N；

When sample format is 4:1:1, $M_{\mathrm{UV}}=\frac{M}{4},$ _NUV＝N；

When sample format is 4:2:0, $M_{\mathrm{UV}}=\frac{M}{2},$ $N_{\mathrm{UV}}=\frac{N}{2};$

Frame-to-frame differences Diff_Frame:

This feature is found the solution poor based on the monochrome information of front and back two width of cloth images, as shown in the formula,

$\mathrm{Diff}\_\mathrm{Frame}=\frac{\underset{i=0}{\overset{N-1}{\mathrm{\&Sigma;}}}\underset{j=0}{\overset{M-1}{\mathrm{\&Sigma;}}}|Y(i,j,t+1)-Y(i,j,t)|}{M\&times;N}---\left(18\right)$

Wherein, variable t is the time shaft parameter of video sequence; This parametric solution of frame-to-frame differences be the average information of the luminance difference absolute value of back one frame and former frame respective pixel.

2. method for evaluating video quality according to claim 1 is characterized in that: the characteristic value Block that calculates total blocking effect _Image, when wherein carrying out fast fourier transform, after the capable first-order difference of Q added up, again the array that adds up is carried out fast fourier transform, and calculates the amplitude spectrum of fourier coefficient; Calculate the capable horizontal direction blocking effect characteristic value of this Q; Do same operation to ensuing Q is capable, the value of Q is 8 to 16 any one numerical value, and Using such method is handled the view picture image, and the characteristic value addition of all horizontal directions is asked on average, is blocking effect characteristic value in the horizontal direction.

3. method for evaluating video quality according to claim 1, it is characterized in that: use wavelet transformation to carry out the frequency domain energy spectrometer luminance picture, the small echo that the frequency domain energy spectrometer uses is the W53 small echo, and its high-pass coefficient and low-pass coefficients are respectively { 0.25 ,-0.5,0.25}, { 0.125,0.25,0.75,0.25 ,-0.125}; Amplitude to the wavelet coefficient after the level Four conversion is added up respectively, promptly tries to achieve the quadratic sum with the wavelet amplitude of one-level, and divided by the total sample of this one-level, and the value that obtains is the ENERGY E (Lx) on the frequency band of this one-level correspondence; X wherein is 0,1,2,3,4;

Utilize the non-linear bandpass characteristics of CSF then, the wavelet coefficient of different spaces frequency band after the wavelet decomposition be weighted, the frequency domain energy calculation as shown in the formula, Fre_Energy is and obtains image frequency domain energy distributions feature:

Fre_Energy＝2.25×E(L0)+2.87×E(L1)+3.16×E(L2)+2.56×E(L3)+1.00×E(L4) (16)。

Images