CN101996406A - No-reference structural sharpness image quality evaluation method - Google Patents

Abstract

The invention discloses a no-reference structural sharpness image quality evaluation method, which comprises the following steps of: acquiring an original image input in a computer; preprocessing the original image, removing influence of isolated noise points on the sharpness of the original image and acquiring an original image to be evaluated; constructing a reference image for the original image to be evaluated through a low pass filter; respectively performing gradient calculation on the original image to be evaluated and the reference image, and extracting sub-image vectors with rich texture information; calculating the structural similarity between corresponding sub-image vectors so as to obtain structural similarity results of the sub-image vectors; and calculating no-reference structural sharpness by using the obtained structural similarity results of the sub-image vectors so as to obtain quality evaluation index no-reference structural sharpness of the original image. The reference image is constructed through an imaging model, no-reference image quality evaluation is performed by a reference image quality evaluation method aiming at image blurring, and the method is applied to the fields of imaging quality detection and control of an imaging system, evaluation of an image processing algorithm, and the like.

Description

No reference configuration sharpness image quality evaluating method

Technical field

The invention belongs to image processing field, particularly the image quality evaluation field can be used for image quality and detects, monitors that the control of imaging system parameter also can be used for the image processing algorithm assessment, image processing algorithm parameter optimization, network image transmission quality assessment etc.

Background technology

Along with the development of infotech, image just more and more comes into one's own as the carrier of information.Image quality evaluation is an important research direction of image processing field, the image quality that can be used for all kinds of imaging systems detects, and guide the control of imaging system, make system always work in optimum condition, also can be used for assessment, parameter optimization of all kinds of image processing algorithms etc., make the comprehensive output result of algorithm reach optimum, also can be used for image network quality monitoring or the like.

Image quality evaluating method is divided into subjective picture quality evaluation method and objective image quality evaluating method.Because the final host of nearly all image all is a human eye, so the subjective picture quality evaluation method can provide picture quality result the most accurately, and the target of objective image quality evaluating method is exactly to provide the image quality evaluation result who meets subjective assessment.The objective image quality evaluating method divides three classes: full reference image quality appraisement method is the method that a kind of difference by image more to be assessed and original reference image provides the image quality evaluation result, is actually the calculating of a kind of image " fidelity "; Partial reference image quality appraisement method does not have reference picture, but the characteristics combination with some reference pictures is estimated principle and preceding a kind of basic identical as prior imformation; And non-reference picture quality appraisement method is meant under the situation without any reference information, and the quality of single-frame images is made an appraisal.During great majority were used, reference image information all can't obtain, and non-reference picture quality appraisement method is the research emphasis and the difficult point in image quality evaluation field.But current various non-reference picture quality appraisement methods are calculation of complex all, and output area is not bounded, so we can't obtain the sharpness information of image from the evaluation of estimate of single-frame images.

Summary of the invention

The problem of technical solution: in order to overcome the problem that current various non-reference picture quality appraisement methods exist, the present invention gets in touch image blurring reason, mathematical model in conjunction with imaging system, a kind of method of image configuration reference picture to be assessed that is has been proposed, thereby utilizing has the image quality evaluating method of reference picture to carry out non-reference picture quality appraisement, proposed at last a kind of at image blurring image quality evaluation index, no reference configuration sharpness NRSS (No-Reference Structural Sharpness).This index is calculated simple, and output bounded, can obtain image definition information by the evaluation of estimate of single-frame images.

Technical scheme of the present invention:

In order to realize described purpose, a kind of step of not having a reference configuration sharpness image quality evaluating method provided by the invention is as follows:

Step S1: the original image that obtains the input computing machine; Original image is carried out pre-service, remove the influence of isolated noise point, obtain original image to be assessed the original image sharpness;

Step S2: by low-pass filter is original image structure reference picture to be assessed;

Step S3: original image to be assessed and reference picture are carried out the gradient computing respectively, extract the informative subgraph vector of texture structure;

Step S4: calculate the structural similarity between corresponding subgraph vector, obtain the structural similarity of each subgraph vector;

Step S5: utilize the structural similarity of each subgraph vector that obtains to calculate no reference configuration sharpness, the quality evaluation index that obtains original image does not have the reference configuration sharpness.

Wherein, described low-pass filter is mean filter or Gauss's smoothing filter, and the ripple door size of wave filter is 5 * 5～9 * 9.

Wherein, it is described that original image to be assessed is carried out the employed gradient filtering operator of gradient computing is the Sobel operator, the Sobel operator is the operator of the texture gradient information of extraction level and vertical both direction, the texture leaching process when meeting human eye to image observation.

Wherein, the subgraph vector extracting method that texture structure is abundant: the gradient image to original image to be assessed carries out piecemeal, obtains 8 * 8 big or small piecemeals, and piecemeal moves 4 pixels at every turn, adjacent block have 50% overlapping; Calculate the variance of each sub-piece in the gradient image of original image to be assessed, the N piece that extracts variance maximum wherein as a result of, the subgraph vector that extracts is designated as X _i, i=1 ..., N; Correspondence position extracts subgraph vector Y in the gradient image of reference picture again _i, wherein the size of the size of N and original image and structural information to enrich degree relevant, N=2 ^m, m=4,5,6,7,8.

Wherein, it is as follows to calculate the step of structural similarity of corresponding subgraph vector:

$l(X,Y)=\frac{{2\mathrm{\&mu;}}_X{\mathrm{\&mu;}}_Y+{\mathrm{<figure-callout\; id="1"\; label="C"\; filenames="HSA00000337206600011.png"\; state="\{\{state\}\}">C</figure-callout>}}_1}{{\mathrm{\&mu;}}_X^2+{\mathrm{\&mu;}}_Y^2+{\mathrm{<figure-callout\; id="1"\; label="C"\; filenames="HSA00000337206600011.png"\; state="\{\{state\}\}">C</figure-callout>}}_1},$

$c(X,Y)=\frac{{2\mathrm{\&sigma;}}_X{\mathrm{\&sigma;}}_Y+C_2}{{\mathrm{\&sigma;}}_X^2+{\mathrm{\&sigma;}}_Y^2+C_2},$

$s(X,Y)=\frac{{\mathrm{\&sigma;}}_{\mathrm{XY}}+{\mathrm{<figure-callout\; id="3"\; label="C"\; filenames="HSA00000337206600011.png"\; state="\{\{state\}\}">C</figure-callout>}}_3}{{\mathrm{\&sigma;}}_X{\mathrm{\&sigma;}}_Y+{\mathrm{<figure-callout\; id="3"\; label="C"\; filenames="HSA00000337206600011.png"\; state="\{\{state\}\}">C</figure-callout>}}_3},$

SSIM(X，Y)＝[l(X，Y)] ^α·[c(X，Y)] ^β·[s(X，Y] ^γ，

L (X wherein, Y) be illustrated respectively in the gradient image and the c (X of original image to be assessed, Y) two subgraph vector X of correspondence position in the gradient image of reference picture, Y is at brightness, contrast and s (X, Y) similarity measurement of structural information aspect, SSIM is the structural similarity of subgraph vector, and α, β and γ are the weights of three tolerance, μ _X, μ _YBe respectively X, the average of Y, σ _X, σ _YBe respectively X, the standard deviation of Y, σ _XYBe X, the covariance of Y, C ₁, C ₂, C ₃Be in order to prevent that three similarity measurement denominators from being the zero less constant of being got.

Wherein, not have reference configuration sharpness NRSS computing formula as follows for described original image quality evaluation index:

$\mathrm{NRSS}=1-\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{SSIM}(X_i,Y_i).$

Beneficial effect of the present invention: the present invention combines the mathematical model of imaging system well, utilize imaging model to construct reference picture simultaneously, carry out at image blurring non-reference picture quality appraisement with the method that reference image quality appraisement is arranged, can be used for the Detection ﹠ Controling of digital imaging system image quality, the fields such as assessment that also can be used for image processing algorithm have vast market prospect and using value.

Description of drawings

Fig. 1 is a method flow diagram of the present invention;

Fig. 2 is the extraction subgraph vector approach process flow diagram among the present invention;

Fig. 3 (a) is the result that the less image of a secondary defocusing amount is carried out quality assessment to Fig. 3 (c);

Fig. 4 (a) is the result that the big image of a secondary defocusing amount is carried out quality assessment to Fig. 4 (c);

Fig. 5 is the evaluation result to an image sequence.

Embodiment

For making the purpose, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in more detail.

Fig. 1 illustrates method flow diagram of the present invention, and no reference configuration sharpness of the present invention (NRSS) is a kind of at image blurring non-reference picture quality appraisement method, and the concrete steps of its method are as follows:

1. image pre-service

At first, if method of the present invention is used for embedded real time system,, can carry out 1/2 down-sampling to original image earlier in order to reduce calculated amount; If the data volume of pending original image own is little, can calculate according to original size.Original image through down-sampling is designated as f ₀(x, y), x wherein, y represents the ranks coordinate of corresponding pixel points in image respectively.To original image f ₀(x y) carries out 3 * 3 medium filterings, median-filtered result be designated as f (x, y).Original image blurs because high fdrequency component is lost and caused, so in fact the fuzzy evaluation of original image is exactly the contained high fdrequency component of computed image what.And therefore isolated noise spot should be removed just corresponding to the high fdrequency component in the original image.Median filter be a kind of denoising effect better and can not lose the wave filter of image detail, so the present invention selects the median filter denoising for use.

2. structure reference picture

The image quality evaluating method of no reference is not owing to there is reference picture, so general algorithm complexity all is not easy to utilization.Therefore, in order to utilize calculating reference image quality appraisement method is arranged simply, the present invention is an image configuration reference picture to be assessed by the low-pass filtering method that is combined into as system mathematic model.By the mathematical model of imaging system as can be known, imaging system is actually a low-pass filter, and the cutoff frequency of wave filter is low more, and the original image that is obtained is just fuzzy more.Therefore, picture rich in detail is abundanter than the high-frequency information of blurred picture.If a width of cloth original image is carried out low-pass filtering, picture rich in detail will lose most high-frequency information, and the reference picture that obtains is thickened; If original image itself is fuzzy, then low-pass filtering is little to its influence.Imaging system has two kinds of widely used mathematical models: disk model and Gauss model.If adopt disk model, then Dui Ying low-pass filter is a mean filter; If adopt Gauss model, then corresponding gauss low frequency filter.In conjunction with subsequent step of the present invention, the size of wave filter elects 7 * 7 as can obtain effect preferably.Structure reference picture f ' (x, process y) can be expressed from the next:

f′(x，y)＝LPF[f(x，y)] (1)

LPF[wherein] be spatial domain low-pass filter template, can be mean filter, also can be gauss low frequency filter.

3. gradient calculation

Image blurring to show as detail textures unintelligible, not bigger variation of image outline.(Human Vision System, result of study HVS) shows that human eye is especially responsive to the horizontal and vertical detailed information of image to the human visual system.And be used for the airspace filter operator that gradient information extracts in the classical just Flame Image Process of Sobel operator.Gradient information based on the Sobel operator carries out as follows:

g _x(x，y)＝f(x+1，y-1)+2f(x+1，y)+f(x+1，y+1)-f(x-1，y-1)-2f(x-1，y)-f(x-1，y+1)(2)

g _y(x，y)＝f(x-1，y-1)+2f(x，y-1)+f(x+1，y-1)-f(x-1，y+1)-2f(x，y+1)-f(x+1，y+1)(3)

$g(x,y)=\sqrt{g_x{(x,y)}^2+g_y{(x,y)}^2}---\left(4\right)$

Wherein, (x y) refers to original image to be assessed, g to f _x(x, y) and g _y(x is respectively an original image to be assessed y) extracts the result at the gradient information of level and vertical direction, to the Sobel operator gradient information of original image to be assessed extract the result be designated as g (x, y).Use the same method to the reference picture of being constructed carry out gradient information extract the result be designated as g ' (x, y).

4. extract the informative subgraph vector of texture structure

As Fig. 2 extraction subgraph vector approach process flow diagram among the present invention is shown, the readability of image is mainly showed by grain details information, therefore is necessary to extract earlier the informative subgraph of image texture, these subgraphs is estimated again.Extracting subgraph among the present invention carries out in accordance with the following methods: to image g (x, y) carry out the piecemeal of 8 * 8 sizes, move 4 pixels at every turn, promptly interblock have 50% overlapping, calculate the local variance of each subgraph, the N piece subgraph that extracts variance maximum wherein is as subgraph vector X _i(i=1,2 ..., N) and the position of writing down the subgraph vector.(x, y) the middle correspondence of extracting is with reference to subgraph vector Y at image g ' to utilize the position of being write down _i, i=1,2 ..., N.The size of N value is relevant with the texture information of image, in conjunction with the characteristics that embedded system is handled, General N=2 ^m, m=4,5,6,7,8.In order to improve extraction rate, the leaching process of subgraph vector carries out as follows:

1) sets up loop variable and be initialized as i=0, j=0; Set up variance array Variance[N] and all be initialized as 0; Set up two-dimensional position array Location[2] [N], and all be initialized as 0, its first line item subgraph row-coordinate, the second line item subgraph row coordinate.Variable Min_v is used for storing the minimum value among the variance array Variance, is initialized as 0, and variable Min_pos is used for writing down the position of minimum variance in array, also is initialized as 0.If picture traverse is Width, establishing picture altitude is Height.

2) calculating location be (i, subgraph variance j), and with variable Min_y relatively:

If this subgraph variance ratio variable Min_v is big, then upgrade variance array Variance[Min_pos] be subgraph variance, the Min_pos row of two-dimensional position array Location be updated to (i, j).Changed for the 3rd step over to.

If this subgraph variance ratio variable Min_v is little, then directly changed for the 4th step over to.

3) traversal array Variance[N], variable Min_v is updated to element minimum in the array, variable Min_pos is updated to the position at least member place.

4) upgrading loop variable is i=i+4:

If i＞Width-8, then i=0 changed for the 5th step over to;

Otherwise, changed for second step over to.

5) upgrading loop variable is j=j+4:

If j＞Height-8 then changed for the 6th step over to;

Otherwise, changed for second step over to

6) according to array Location[2] [N] positional information of providing, (x extracts subgraph vector X in y) at image g _i, i=1,2 ..., N is at g ' (x, y) the middle subgraph vector Y that extracts _i

5. calculate the structural similarity of corresponding subgraph vector

If X, Y are the subgraph vectors of one group of correspondence, then their length all is 64.Calculate separately average and variance, calculate the covariance of two vectors again, be designated as respectively: μ _X, μ _Y, σ _X ², σ _Y ², σ _XYThe similarity degree of compute vector aspect brightness, contrast and structural information measured respectively again, carries out according to following formula respectively:

$l(X,Y)=\frac{{2\mathrm{\&mu;}}_X{\mathrm{\&mu;}}_Y+{\mathrm{<figure-callout\; id="1"\; label="C"\; filenames="HSA00000337206600011.png"\; state="\{\{state\}\}">C</figure-callout>}}_1}{{\mathrm{\&mu;}}_X^2+{\mathrm{\&mu;}}_Y^2+C_1}---\left(5\right)$

$c(X,Y)=\frac{{2\mathrm{\&sigma;}}_X{\mathrm{\&sigma;}}_Y+C_2}{{\mathrm{\&sigma;}}_X^2+{\mathrm{\&sigma;}}_Y^2+C_2}---\left(6\right)$

$s(X,Y)=\frac{{\mathrm{\&sigma;}}_{\mathrm{XY}}+{\mathrm{<figure-callout\; id="3"\; label="C"\; filenames="HSA00000337206600011.png"\; state="\{\{state\}\}">C</figure-callout>}}_3}{{\mathrm{\&sigma;}}_X{\mathrm{\&sigma;}}_Y+C_3}---\left(7\right)$

C wherein ₁, C ₂, C ₃Be in order to prevent that denominator from being the zero less constant of being got.(X Y) is example, and if only if μ with l _X=μ _YThe time, the similarity degree of two vectors aspect brightness just is 1, and μ _XWith μ _YDifference big more, (X, value Y) will be more little for l.

Structural similarity calculates according to following formula:

SSIM(X，Y)＝[l(X，Y)] ^α·[c(X，Y)] ^β·[s(X，Y] ^γ (8)

Wherein α, β and γ are the weights of three tolerance, generally speaking α=β=γ=1.

6. calculate the no reference configuration sharpness of original image to be assessed

Narrated in the step in front, picture rich in detail has lost most of high-frequency information through after the low-pass filtering, and therefore the structural similarity between the corresponding subgraph vector that extracts is low.And there is not reference configuration sharpness NRSS is a kind of boundedness tolerance that is directly proportional with image definition, and therefore, the computing formula of no reference configuration sharpness NRSS is as follows:

$\mathrm{NRSS}=1-\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{SSIM}(X_i,Y_i)---\left(9\right)$

Obviously, image is clear more, and the output valve of no reference configuration sharpness NRSS is high more; Image is fuzzy more, and the output valve of no reference configuration sharpness NRSS is low more.Simultaneously because the output boundedness of structural similarity SSIM, the output of no reference configuration sharpness NRSS also be between [0,1) between.

As Fig. 3 (a) result that the less image of a secondary defocusing amount is carried out quality assessment is shown to Fig. 3 (c); Wherein Fig. 3 (a) is an image to be assessed; Fig. 3 (b) is the reference picture of structure; Fig. 3 (c) extracts the result for subgraph; The NRSS method is 0.9136 to the evaluation result of Fig. 3 (a);

As Fig. 4 (a) result that the big image of a secondary defocusing amount is carried out quality assessment is shown to Fig. 4 (c); Fig. 4 (a) is an image to be assessed; Fig. 4 (b) is the reference picture of structure; Fig. 4 (c) extracts the result for subgraph; The NRSS method is 0.5694 to the evaluation result of Fig. 4 (a);

As Fig. 5 evaluation result to an image sequence is shown; To same target imaging, the conditioning equipment lens location makes the gained image burnt to standard from out of focus to this image sequence continuously, arrives out of focus again by same equipment.

The above; only be the embodiment among the present invention; but protection scope of the present invention is not limited thereto; anyly be familiar with the people of this technology in the disclosed technical scope of the present invention; can understand conversion or the replacement expected; all should be encompassed in of the present invention comprising within the scope, therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.

Claims (6)

1. a no reference configuration sharpness image quality evaluating method is characterized in that, comprises that step is as follows:

Step S1: the original image that obtains the input computing machine; Original image is carried out pre-service, remove the influence of isolated noise point, obtain original image to be assessed the original image sharpness;

Step S2: by low-pass filter is original image structure reference picture to be assessed;

Step S3: original image to be assessed and reference picture are carried out the gradient computing respectively, extract the informative subgraph vector of texture structure;

Step S4: calculate the structural similarity between corresponding subgraph vector, obtain the structural similarity of each subgraph vector;

Step S5: utilize the structural similarity of each subgraph vector that obtains to calculate no reference configuration sharpness, the quality evaluation index that obtains original image does not have the reference configuration sharpness.

2. the method for claim 1 is characterized in that, described low-pass filter is mean filter or Gauss's smoothing filter, and the ripple door size of wave filter is 5 * 5～9 * 9.

3. the method for claim 1, it is characterized in that, it is described that original image to be assessed is carried out the employed gradient filtering operator of gradient computing is the Sobel operator, the Sobel operator is the operator of the texture gradient information of extraction level and vertical both direction, the texture leaching process when meeting human eye to image observation.

4. the method for claim 1 is characterized in that, the subgraph vector extracting method that texture structure is abundant: the gradient image to original image to be assessed carries out piecemeal, obtains 8 * 8 big or small piecemeals, and piecemeal moves 4 pixels at every turn, adjacent block have 50% overlapping; Calculate the variance of each sub-piece in the gradient image of original image to be assessed, the N piece that extracts variance maximum wherein as a result of, the subgraph vector that extracts is designated as X _i, i=1 ..., N; Correspondence position extracts subgraph vector Y in the gradient image of reference picture again _i, wherein the size of the size of N and original image and structural information to enrich degree relevant, N=2 ^m, m=4,5,6,7,8.

5. the method for claim 1 is characterized in that, the step of structural similarity of calculating corresponding subgraph vector is as follows:

$l(X,Y)=\frac{{2\mathrm{\&mu;}}_X{\mathrm{\&mu;}}_Y+C_1}{{\mathrm{\&mu;}}_X^2+{\mathrm{\&mu;}}_Y^2+C_1},$

$c(X,Y)=\frac{{2\mathrm{\&sigma;}}_X{\mathrm{\&sigma;}}_Y+C_2}{{\mathrm{\&sigma;}}_X^2+{\mathrm{\&sigma;}}_Y^2+C_2},$

$s(X,Y)=\frac{{\mathrm{\&sigma;}}_{\mathrm{XY}}+C_3}{{\mathrm{\&sigma;}}_X{\mathrm{\&sigma;}}_Y+C_3},$

SSIM(X，Y)＝[l(X，Y)] ^α·[c(X，Y)] ^β·[s(X，Y] ^γ，

L (X wherein, Y) be illustrated respectively in the gradient image and the c (X of original image to be assessed, Y) two subgraph vector X of correspondence position in the gradient image of reference picture, Y is at brightness, contrast and s (X, Y) similarity measurement of structural information aspect, SSIM is the structural similarity of subgraph vector, and α, β and γ are the weights of three tolerance, μ _X, μ _YBe respectively X, the average of Y, σ _X, σ _YBe respectively X, the standard deviation of Y, σ _XYBe X, the covariance of Y, C ₁, C ₂, C ₃Be in order to prevent that three similarity measurement denominators from being the zero less constant of being got.

6. the method for claim 1 is characterized in that, it is as follows that described original image quality evaluation index does not have reference configuration sharpness NRSS computing formula:

$\mathrm{NRSS}=1-\underset{i=1}{\overset{N}{\mathrm{\&Sigma;}}}\mathrm{SSIM}(X_i,Y_i).$

Images